Code ExplodeGreat software can make life better

I’ve been playing with Apache Hadoop lately to get my head around MapReduce and Distributed File Systems. Its quite an interesting subject. Facebook, Google, Twitter and all the big players of the internet use the techniques to scale at an insane level. However the issue we are finding is that they are closed systems. Unlike email which had a POP protocol, a gmail user can send emails to hotmail users and vice versa, most of the web systems are closed and don’t seem to work well with each other.

One of the other big problems is that machines don’t know they are machines and they can talk to other machines. Programmers still need to code custom things, machines aren’t able to self program themselves and adapt their behaviour to make our lives more streamlined.

The worst part is every application comes with a “Help” manual but normal users can never figure out how to use help to solve their problem. Google does a far better job. Are our books and the way we write text manuals flawed in a way that we don’t yet understand how to properly represent knowledge and compute it ? But even google is naive. Google is a crawler and indexer. They can give us a list of pages where the information exists, unlike wikipedia or stackoverflow which allows generation of massive amounts of organized data.

My gut feeling is that to make the semantic web work, we need to somehow understand how knowledge works, how we can efficiently represent insane amounts of data and be able to compute it to make smarter decisions.

At the company where I work “RocketBoots”, every project presents a new challenge. From Web Programming with HTML/CSS/JS, to Mobile Programming on Android, Objective C, to RIA’s with Flex and C# to serverside CF/PHP to embedded development with Python/C/C++ to Highly distributed programming with erlang and hacking languages such as LISP and Clojure. I am one of those grateful that gets to work across a range of programming languages and problems.

I’ve been noticing a trend though. I think there is a fundamental problem that almost all code editors and compilers treat code as text. But in fact all code we write is a structure. Its like a tree. The IDE’s that do understand the language like Eclipse allow magnitudes more productivity in getting things done. Imagine 10 years down the lane, what would it be like to program ? I think one day everyone will be programmers, but not with the programming languages we have currently.

As it was mentioned in the post before

People don’t want computers or software, they want what computers and software can do.

I’ve been thinking about this problem a lot lately. When I learnt coding we had a text editor like vi in which we wrote code. It was painful, I had to check the documentation on what function accepts what parameter. Then I found eclipse, which had all the autocompletion mojo and I soon found myself debugging more code rather than fixing compiler errors. Eclipse had a model of the grammar, it would know all the possible things that could go in place of the keyboard cursor. I think the human brain is better left to solve problems and the computer can be left to use its 2 billion ops a second power for predictive analysis on what the user is going to type next. I have yet to see a code editor with serious machine learning algorithms that can learn from billions of lines of code in open repositories and figure out a language’s EBNF structure, figure out what is the probability of a function being used and offer a great experience to the coder.