Compared to when I started in the early 1980's, developing applications today requires way too much effort and knowledge. Shouldn't our profession have found ways to make things easier by now?
My background is both in desktop applications and for the last 7 years, web applications. Especially for the latter, the number of technologies necessary for development usually exceed the ability of most programmers to master them all. Often today people are forced to specialize in front-end or back-end or middle tier and are unable to function if required to do something else. The ability to investigate, comprehend and then switch technologies is getting rarer too as larger companies "standardize" on a set of frameworks, systems and languages to try and avoid the overload on their programmers. This natural reaction then tends to create stale inflexible project teams. In the long run such limited choices on technologies then keeps the teams from learning new technologies that actually might make things simpler.
As an example, right now in order to develop a web app from front to back using J2EE requires some or all of the following knowledge:
- Java
- Servlet Spec
- JSP
- Struts
- XML
- HTML
- CSS
- Javascript (today also Ajax frameworks like Prototype)
- EJBs or Hibernate or JDO or iBatis or other ORM mapping framework
- SQL (at least query unless you also are expected to design databases, then DDL)
- JUnit or other test framework
- Container specifics for Weblogic / Websphere / Tomcat / JBoss etc
- Deployment descriptors and details
- Web server configuration
- Security frameworks like Acegi, etc
- Caching technologies
- Other J2EE stuff like JMS
- Other utility frameworks like Apache Commons.*, etc
- Optional container frameworks like Spring or the various Portals
- If you're insane, maybe XSLT
I've probably forgotten something here. This is insane, how can anyone possibly be able to not only understand the basics of all these technologies, but be able to correctly design, code, assemble and understand how to test using all these various bits. It's no wonder people either specialize, move to alternate languages (like PHP or Python which have their own set of frameworks and knowledge requirements), or just muddle through and make barely or non-functional crap. The funny thing is from reading job ads, you are expected to be able to know all of these technologies by heart and be able to code in the dark with one hand cut off (not to mention 7 years of experience at each). If you finally start working at the company in the job ad you find out that virtually no one there can even master more a handful of the technologies.
So what is the answer? Ruby-on-Rails, PHP, Python, maybe functional languages like ML, Lisp, Haskell? I'm not sure there is a radically better solution that we could all easily switch to. As much as I like Java you have to work really hard to make it simple, and even then the web technologies will trip you up. The whole stack from data up to browser is still a nightmare of moving parts. Somewhere out there is a set of solutions that will actually make our lives easier, allow developers to focus on the business of meeting customer's needs without focusing on the damn technology muddle that we call Programming today. I sure hope to see that day some day.

vm 12/18/2006 04:55
And also the developer should be know all the versions of these technologies. (according to job ads ) :-)
vm 12/18/2006 04:57
And also the developer should know all the versions of these technologies. (according to job ads ) :-)
sorry wrong sentence :-)
Steve 12/18/2006 06:22
I think your statement about the large set of skills needed to develop a web app in Java is out of date, and the comparison with frameworks in other languages isn't right.
The problem with your comparison with other frameworks is you aren't comparing like with like. You can't compare what would be a serious enterprise application (involving JMS and major security frameworks) with PHP or Ruby on Rails. The equivalent to those would be a lightweight JSP application, perhaps using some taglibs for data access. There really is no equivalent to the full J2EE stack in those languages.
Also, even with alternatives to Java, there is a lot to deal with - these aren't just J2EE issues. There is still deployment, web server configuration, HTML, CSS, security issues, session handling, testing, SQL, some testing framework, Javascript and cacheing.
You don't need all those J2EE skills even for pretty major website development in Java. There are new Java frameworks that pretty much handle most of what you need, like Seam.
Bart 12/18/2006 08:05
I completely agree with the article. I remember dabbling in programming in the early 90's (middle school) and all the books that were out were specific to syntax/hardware/physics etc. There was no internet, business intelligence, networking, advanced graphics, widely used database software etc. to learn in order to be considered a good programmer.
This is why the quality of the code has gone down slightly and OOP design with UML is not widely used as before. You have programmers doing 10 things, where they only might be really good at about 2-3 and the rest they get by.
Penguin Pete 12/18/2006 08:47
But compared to the early 1980's? How much development for the World Wide Web did you do then? You must have been way ahead of the curve!
Programming is more difficult today because computers have a lot more functionality. They can do more, so we have to control more of what they do.
That said, I've also said that the state of programming for the web is dire. It's time for a new Big Language that does it all. But then I remember Perl, the last Big Language, and how you can study Perl for a lifetime and a half and not master all of it. So our choice is that, or lots of little languages.
Jan 12/18/2006 09:17
I'm not defending overly complicated frameworks, but things wasn't that nice back in the day as you like to remember... Lots of very basic stuff had to be coded from scratch because of primitive languages and poor libraries.
Simon 12/18/2006 09:22
If it was easy anyone could do it.
CM 12/18/2006 10:13
Wanted English speaker:
Must be able to speak in all modern versions of english including pig-latin. Must be familiar with all vernacular including MTV versions of street lingo. Must have lived in an inner-city area of at least 15 large American cities and three British ones possibly two Australian. Candidate must be able to speak standard english without an accent the way most television announcers do.
Let's not forget that any candidate must be able to write in all the differenet english grammers and know the country references to "colour" as vs "color". blah blah blah...
Apply all this to a job market for web programing and things really suck. I agree.
Matthew Cornell 12/18/2006 11:05
I agree! Esp. when trying to do a mashup, pull info from a page, etc.
Codist 12/18/2006 11:08
In the mid-80s all I needed was Inside Macintosh, a K&R C book and maybe a 68000 assembly language manual. With that I (and various team members) build Trapeze, Persuasion and Deltagraph.
Talking with my friends in the java corporate world, and reading the job ads, that list above still represents much of the java web application universe. But it is sad.
john 12/18/2006 12:14
Is it even possible to write a scalable, enterprise grade web application using only technologies of the early '80s?
If your current project can be developed using the tools/technigues of the early '80s, then its certain that current tools/techniques would appear to be unnecessary. Perhaps a case of using the wrong tools for the job.
As I remember the early '80s, timeshare operating systems were new, GUI apps were new, OO techniques were new, relational databases and transactional processing were new, distributed applications were new. There was very little in terms of software engineering techniques and the most important class a CS major would take is compiler design. There were hundreds of development and runtime environments, languages and libraries.
I think there really was alot to learn and know in the early '80s. Trouble is, nobody learned it yet.
Jim 12/18/2006 13:12
I think the real problem is not the number of technologies one must know, but rather the complexity behind each technology that one must know. Unfortunately, many of these technologies were designed to be flexible and used in many different scenarios (not always a bad thing of course). Unfortunately, all of this flexibility leads to overly bloated interfaces. For instance, one must learn a fairly complicated set of interfaces to map objects to the database and perform CRUD operations on them using Hibernate or JPA. Most the complexity in this example is due to the fact that Hibernate needs to be flexible. What if its' users want to use Strings for primary keys, or what about composite keys? What if users need to map to tables that don't follow any conventions? There are so many considerations these enterprise ORM tools must take into account.
If you compare the Hibernate example to persistence in say Ruby On Rails using ActiveRecord, you'll notice there is one huge thing missing to some extent: flexibility. ActiveRecord assumes a certain structure to your database tables. If you conform to that structure, you'll be able to leverage all of its power with little work. Of course you can make ActiveRecord work with tables that don't quite conform to its imposed structure, but then the amount of work you have to do approaches the amount of work for Hibernate.
In some senses, open source Java stacks are beginning to see the light. Spring 2.0 has begun adopting some reasonable defaults in their MVC framework for instance. I'm hoping that this practice will eventually catch on to other technologies such as ORM. Wouldn't it be nice if Hibernate could figure out that your entity's "firstName" attribute maps to a database table column called "first_name"?
I picked on Hibernate and enterprise ORM, but it's easy to find similar examples in the other technologies you listed about where overly flexible interfaces are unnecessarily complicated. If some of these interfaces started to adopt reasonable conventions (and documented them accordingly! :), we'd be a much closer to having technologies that take significantly less time to leverage.
Helder 12/18/2006 14:45
After some years, Internet will be so fast like a LAN. Lots of technology that began (and still exists) from slow net will be unnecessary.
Codist 12/18/2006 15:02
BTW: I am a java programmer and have actually used most of the above technologies to build complex web applications. Currently I use my own web framework, iBatis, H2, Jetty and various apache projects and ajax frameworks for all my projects.
TheMCP 12/18/2006 21:16
Simplifying this mess of divergent technologies is what Water is for:
http://www.waterlanguage.org
Water is an object oriented language with syntax based on XML. It unifies the functionality of XML, HTML, CSS, and more. In Water, and function can be a web service, and any object class or instance can be a web application.
Also, Water will handle all the messiness of AJAX for you, so programming with AJAX in Water is now easier than programming with any traditional form submission/response methodology.
Tom 12/19/2006 02:53
Check out UnCommon Web.
Mike 12/19/2006 03:56
I think Joel On Software explained this problem very simply. Check out this link:
<a href="http://www.joelonsoftware.com/items/2006/12/05.html">Link</a>
Snake 12/19/2006 13:23
Programming isn't hard. You're just stupid.
Brian 12/19/2006 13:39
Are you kidding me? Programming was a lot harder in the pre-Internet days (80's). Today for any really difficult problem I either use a good search engine or a good forum / wiki to figure out problems 24/7.
The widespread popularity of open source helps a lot too, since I don't have to repeatedly reinvent the wheel.
Things are a lot different today - for the better...
William 01/04/2007 21:52
15 - 20 years ago you would not have been able to produce the systems of today . The limited range of tools ( ASM and microcode ) would have made is a very time consuming and expensive undertaking if not almost impossible. The diversity of tool available to the software developer today have made the task very much easier and more productive.
Klaus Meffert 01/05/2007 11:11
Although things are getting more complex and it's now possible doing complex stuff quite easily (such as asynchr. communication of web sites with a server) that wasn't possible in the earlier days, I see programming way too complicated.
IMO, it could be a lot easier. The problem seems to me related to machine translation in its difficulty. How can a machine translate a general-domain text from one dedicated language to another in an acceptable quality? Answering this question is in reach, current approaches consume mass data combined with translations worked out by human translators.
As a programming language is much easier to interpret by a machine, we will soon get more help by our IDEs. This is what I am sure about.
Later on, programming will either become superfluous because of higher-level techniques (of which I think they are not suited nowadays to be used in a wide range of industry software projects, including legacy ones). Or programming will be assisted much more by IDEs.
Saying it is just OK as it is today indicates an inability of imagination.
NXavier 01/18/2008 16:24
I can't stop chuckling over the first comment. I can't count the number of times I've said the same thing after looking at job ads for developers. What, they want the uber-coder superman?
And don't even get me started on head-hunters.
Whatever happened to companies that are just looking to hire people that can write software?