The hottest new skill for all isn’t quite as easy as Codecademy makes it seem

I recently wrote a python script to capture posts and their comments from Facebook and report on the volume, sentiment and topics of discussion. As you’ll quickly realise, my coding skills are self taught through online education platforms such as Codecademy and my ability to write fully functioning tools is still fairly new. The jump from ring-fenced e-learning to staring at a blank text file has been monumentally difficult and taken a large amount of time, but nonetheless I’m really excited to show off my work even if its not that good!

Back Story

I was showing off Socialnaut, a new Social Listening tool created at Black Swan, that covers a huge number of data sources, is quick and responsive to use and covered all the main areas you want to find out about from social media. Within a few minutes I was asked why Facebook wasn’t showing as a data source. This comes up frequently so it was easy to rattle off how Facebook is a private social network and hides most posts from toolsets like ours. Still, they were very interested in using the same analysis techniques such as sentiment and topics on comments to posts on their Facebook page. I knew at Black Swan we had created APIs that I could use for the enrichments so all I would need to do is get the posts and comments from the Facebook Graph API, pass them through the enricher APIs, save to a database and write a few SQL reports to extract the information. It didn’t sound too hard and since we were short on development resource, I thought I’d give it a go myself.

Act 2

I found a great series of posts with full instructions and walk through at http://simplebeautifuldata.com/2014/08/25/simple-python-facebook-scraper-part-1/ so it seemed like I should be able to wrap it all up in double quick time. Alas, that was not the case. I started by installing MySQL which went fine until I tried to connect to the server and couldn’t work out why the password I’d been given during installation was apparently now expired. Anyways, I found out how to reset the password and carried on. Through terminal I connected with the server, created my first test database and things were looking good on the database front. Next up, Python script.

After enjoying the experience of Codecademy and subsequently struggling with plain text editors after being guided to them as the ‘pure’ way to code, I went on the lookout for an IDE that would remove some of the switching between a text editor, terminal and database client, hopefully making things a bit easier so I could focus purely on the code. I found PyCharm, liked the look of it and it got the thumbs up from people on Quora so I took the free trial and I have to admit its probably the reason why I’ve managed to complete this project where others have fallen.

1-Fax_FsXloFBjhyMFliOj6w
PyCharm, my saviour and a great logo

I’ve previously used virtual environments for Python and so went about setting one up within PyCharm and so begins a whole day of struggles and exasperation. Setting up the environment was easy, PyCharm does it all nicely through a GUI, but when it came to MySQL I couldn’t get it to connect. I tried numerous suggestions online but to no avail, after a couple of hours I decided to call it quits and run everything on my main environment where MySQL connected immediately.

So finally it came to actually getting some code into the window, I copied across the base script from the second step of the tutorial and tried running it…error… of course, I need to set up the database! Foolish me! Not so useful was the database set up had been handled in a separate tutorial on the same site, but the homepage gave a 500 error, so it took a little creative Googling to get what I wanted. Anyway, with the database sorted I gave the script another whirl…error… Facebook telling me this needed to be an authenticated call. It looks like the author may have been using an older version of the Facebook Graph API which has been out of use since April 2015. No worries, the tutorial had an authenticated call for a later stage so I could just look at that and adapt the page details function to support it. And so began my voyage into what appears to be true coding; copying bits of code, integrating into my own code and then a relentless stream of working out what broke by trying to find the right way to phrase questions on Google, hoping that some smart, useful person on Stack Overflow has had the decency to answer such a basic problem before.

Culmination

Without taking you through every pitfall, error and stupid blindness to the obvious issue and boring you beyond all belief. I have some advice to people new to coding and those that want to make it easier for them.

  1. CODING IS HARD! It takes perseverance and adults are not normally used to this amount of failure. It is a fantastic skill to learn and can help you in lots of ways but you have to be willing to put in the time. Be prepared to spend hours looking into an issue and then the fix be something as simple as the wrong punctuation at the end of a line.
  2. Development environments are a pain in the arse. Installing everything you need takes time and you need to manage different environments for every project. Also, always deferring to ‘sudo’ when things don’t work first time feels dangerous and probably is.
  3. Error messages are not clear, please tell me how to fix! I understand how error paths have to be very generic, but please try to make them more readable. This also stands for most developer written public facing error messages (form validation, I’m looking at you).
  4. E-Learning tools such as Codecademy have really good UI’s that make it easy to write code but they make the jump to real tools difficult. This in itself isn’t a bad thing but make sure to make guides to make the transition over, at the moment its no mans land!
  5. Debugging should be taught, it is a skill unto itself.

Thanks for reading, let me know your downfalls with learning to code!

This post was first published at https://medium.com/@ashmccallum/why-so-many-people-give-up-learning-to-code-3166c240f9d0#.34k3jkqpq 

A cygnet from the very earliest days of Black Swan, Ash worked extensively on creating the Nest analytics platform and if you ask nicely he might even show you his black swan tattoo!