Home About The Codist RSS Feed

Writing A Web Application Framework, One More Time, Again
Sep 18, 2007 10:52 perm link Readers: 1683

In the past year I've written two complete java web application frameworks from scratch; one like Wicket, and one (Blogfiche) a more traditional MVAC framework with an alternative syntax to HTML. So why am I building another one?

Am I nucking futs?

The answer is always the search for more programmer productivity, flexibility and reliability. Blogfiche (which runs this blog and other things) has proven itself to be flexible, reliable, and for what I use it for, scalable. Digg and Reddit traffic don't faze it, even though it runs on a shared (and rather overloaded) server. The alternate HTML syntax is fun to use, much less irritating than HTML if you write by hand:

    if(headlines:sizenot0)[
      div(class=articlebox heads)[
        loop(post|headlines)[
          p[a(href={post:url})[{post.title}] span[{post.created:datetime}]]
        ]
      ]
    ]

So why do another one? Mainly I want to be able to build an entire web application without any recompilation, yet still have good performance in production. I am using the solid foundation of my Fiche framework (built on the servlet spec) including its automatic caching of web content. On top of this I intend to support Javascript (Rhino) and Groovy (and anything else interesting) and see what I like better. Although I can use the alternative HTML syntax, I am building a view technology on top of regular XHTML to make it easier to use examples from the web.

Something like:

  <head>
    <title>{title}</title>
  </head>
  <body>
  {if(test)}
    <p>Test</p>
  {else}
    <p>Not Test</p>
  {end}
  </body>

For database support I will continue to use iBatis, if I can properly reload the sqlmaps when they change, or maybe try something new like couchDB, or even build something similar to ActiveRecord. There is a lot of possibilities here; the primary point is to be able to update code and database use without compilation.

I am not trying to replicate PHP or ROR or whatever flavor you might enjoy. Software is an infinitely malleable clay; you don't know what you can do unless you try something.

Again with the why? Why not, I enjoy it, I like productivity, challenges, and I have time while I wait in absolute contractor hell; at least my mind is working and I can avoid going absolutely bonkers waiting on people to make a #@!$#@decision.

My Tags:

  • florin: Sep 19, 2007 04:35

    Have a look at JPublish. Use the Scripting action support and you don't need to recompile :)

    As for the UI dev part, we are supporting: Velocity, Freemarker and StringTemplate :)

    http://code.google.com/p/jpublish/

    Let us know your impression,

    -florin

  • Mark Menard: Sep 19, 2007 06:29

    Hi,

    You said, "Mainly I want to be able to build an entire web application without any recompilation, yet still have good performance in production." That's what drove me to work on what I call Groovy Works, http://code.google.com/p/groovyworks/ and http://www.vitarara.org/cms/taxonomy/term/70.

    It's based on Struts 2, which is based on XWork. It integrates with Spring, and thus with everything that Spring supports. I can write my actions, service beans and DAO's in Groovy and just keep coding. My service beans and DAO's need to implement an interface, so my basic work flow is write the interface and an empty implementation of the interface. Then I wire my DAO, service bean and action together. At this point they do nothing, but they are wired. From that point on I can code and reload with no app server restart. Write my JSP/Velocity/Freemarker template, action, business logic and data access without the usual compile, deploy, restart cycle.

    Take a look,

    Mark

  • Mike Seth: Sep 19, 2007 13:08

    No, no and no!

    Frameworks are not born out of an abstract desire for productivity, flexibility or reliability. These are secondary outcomes caused by many factors, and the framework is not necessarily a substantial part of them: if your broken regular expression code crashes the PHP interpreter, or your JVM runs out of virtual memory, your framework of choice may as well silently die.

    Frameworks are written so that the programmer is comfortable expressing his or her vision in idioms that suit the problem domain space; that is, they provide an environment that fits what you're doing, and allows you to do so consistently. A framework is not, by a long shot, an extension of your code but a constraint on it. If you accept it, the framework pays you with consistency and bribes you with library services. If you reject it, benefits of the framework are nullified, and you might as well glue some 3rd party libraries with a bunch of spaghetti code and call it a day.

    Hint: RoR is not a framework.

  • Joe: Sep 19, 2007 14:34

    Perfectly sensible - I've yet to use a framework (in any area, including web), that doesn't contain at least a few pain-points. There's nothing duller than trawling through someone else's code trying to figure out why something that the framework should actually make a trivial task is harder to accomplish than if you weren't using a framework at all.

    There's no better way to understand why frameworks are the way they are than to build one yourself. After all, the big web frameworks started out as a single-person project, usually to address a specific type of application space, so there's plenty of room for different approaches for different application needs.

  • Add Comment

Kindergarten, The Digg Effect, Shared Hosting and Traffic Rankings
Apr 09, 2007 09:40 perm link Readers: 1038

Hows that for a complicated title?

I posted All I Need To Know To Be A Better Programmer I Learned In Kindergarten last Thursday morning, and watched it do absolutely nothing that day (600 readers or so, way below an average day).

The next day it was added to Reddit and during the day had about 6,000 readers (good, but no record). That night I went to Good Friday services and shared a meal with friends. Before going to bed (late) I check the stats and noticed 2,000 people had read the article in the last hour. I saw that it had 69 diggs so I visited Digg to see where it was. It turned out it was #2 on Digg. This was the first time one of my articles went anywhere on Digg (usually Reddit is my main traffic site). In the next two days I had nearly 40,000 readers and over 1,000 diggs.

This site runs on a shared host on linux (some kind of mid-level single CPU Athlon box) at kattare.com with apache on the front end passing all traffic to my jetty server, which runs my java blog application (BlogFiche) on top of my own web application framework (Fiche) using H2 as the database. This instance of jetty is also running some other (lower traffic) sites. I wasn't remotely afraid of the "Digg Effect" since I knew exactly how much traffic this platform was designed for. I wouldn't get worried until traffic hit 5 per second or so sustained (that's 18K+ per hour). The max I saw was probably around 3K per hour, so the system was still mostly idle.

My site caches all generated page content and only regenerates it if something changes (I edit the site or a comment is posted). All static content is served via the jetty default servlet, which also does caching. Jetty itself uses Java NIO (non-blocking IO), so it's extremely efficient at serving web content. So in effect the only thing happening on 99.9% of all connections is IO. Digged? No problem.

Of course I wouldn't advertise myself as knowing and solving performance issues if my own site performance sucked. Seeing blogs getting digged and die is a little irritating to people; I can understand a graphics or video site getting swamped, but a simple blog shouldn't have any trouble with 1 connection per second (3,600 per hour).

The funny thing about the traffic is the difference between Google and Alexa. Even though I had 40,000 readers for this one article, my Alexa ranking dropped a lot over the same timeframe. Google's index is filled with references (and outright copies!) of the article, but Alexa shows nothing happened at all. Maybe fewer people are using the Alexa toolbar these days.

I tend to look at my adgridwork account for quick updates to my traffic. Although I am collecting some stats I have been lazy to update the admin portion of this blog so I can see (and show) actual counts on the articles.

It was pretty cool (and humbling) that so many people read the article from all over the world. The web is an amazing place.

My Tags:

  • codist: Apr 09, 2007 11:45

    Another blog posted a similar article at Surviving the digg effect. Unfortunately he hit the digg front page twice at the same time and it leveled his server. Ouch.

  • Dan: Apr 12, 2007 13:26

    Congratulations! It was a great article!

  • Add Comment

The Fiche Framework - Part II
Oct 25, 2006 20:17 perm link Readers: 451

In the second part of my description of Fiche, I want to focus on the benefits of AFL, the markup language for the framework.

The syntax of AFL is much simpler and more concise that HTML or XML, and in practice has been very pleasant to work with.

The basic element is the tag (not a good word but I haven't found a better term yet):

tag[]
tag()[]

The two forms are similar but the first has no parameters (or attributes if you like); the second can have multiple comma-delimited key-value pairs plus a single key which is assumed to be an id. Tags come in two flavors, output tags (that generate content) and command tags (the are executed). All tags can have children (I make no attempt at enforcing illegal combinations). Data references (in the usual dot notation) are allowed anywhere text can be placed including the parameters. The parameter to a command tag is assumed to be a data reference. Data may be followed by an optional : and a decorator (a single valued function). That's the whole syntax.

Examples:

div(exceptionreport,style=display:none;)[]

if(requestinfo.session.authorLoggedIn)[
  a(class=button,href={requestinfo.baseURL}{site.path}/edit)[Add Article]
]

div(post)[
  {post.display}
  p(tags)[Technorati Tags: {post.tags:technotags} ]
]

li[a(class=button,onclick=return addcomment({post.threadId},this))[Add Comment]]

Currently the tags map mostly to actual HTML tags but in the next version I will support more complex tags that could, for example, generate the entire navigation for a site. Decorators are already user definable (and quite handy). As you can see, it is quite readable; however with syntax hiliting in TextMate, it's even better (I do have to update the syntax color a bit, it's a little out of date)

Of course for those of you reading this, its somewhat like reading about the surface of Mars; it's interesting, but you can't go there yet. At this point there is no reason to release the framework as it is still very young (though quite functional). As it becomes closer to being completed, and anyone really cares, I will release it as open source. I am not in framework business (reading this blog lately you would think so); once this thing is fairly well done I can start on the real work.

Fiche in a nutshell is a web framework combined with a lightweight CMS backend, all built on standard java technologies, using a simplified web command language named AFL.

My Tags:

The Fiche Framework, Part 1
Oct 23, 2006 21:12 perm link Readers: 450

Fiche is my latest framework, the initial (working) version taking about 3 weeks of development to get to the stage you see. At my previous employer I wrote a Wicket-like framework in about 2 months which was abandoned when I left the company to pursue my own start-up.

I learned a lot in having done this before (actually I have written a couple other partial web frameworks in the past), in addition to working with a whole slew of web frameworks over the years. What I want most in a web framework is:

  1. ease of design/development
  2. minimal configuration
  3. good performance

Of course many web frameworks claim to have these as well, and many do; however none actually excited me other than WebObjects, which is about the oldest web framework in existence. In the past it has always cost too much money for deployment (though I believe it will become free in Leopard). Even WO has the same difficulty with connecting program information to HTML as most frameworks do, and I decided to do something different.

Simple to me might confuse the heck out of someone else but I am writing this for myself first before I even think of getting anyone else involved.

Fiche is built on a fairly normal basis in Java (5) and has the following technology stack:

I used the H2 database to store all the site artifacts except static stuff (like images) and currently the AFL data files (for now). The database has many CMS features in place, some not quite fully developed in this early version. In the schema are the usual person/role/permission management, sites, pages, content and misc tables. The database supports versioning and staging, two basic CMS features, currently only minimally supported in the code but fully designed. I have also implemented basic support for Jackrabbit Repository, but decided to stick with the database for now. Assets such as images and the like will be stored on disk with references in the database.

I spent a year or so as a Documentum Architect so the whole CMS idea is nothing new to me.

iBatis is my favorite choice for an ORM solution, mainly due to its low level approach which allows me to easily use SQL without compromise. I automatically generate the iBatis Mappings and Java data objects from the master sql file. Hibernate is everyone's darling these days, but for my taste it has way too much configuration and features, and gets in your way far too much. I feel the same about Spring, which took a simple idea and created a monster.

If you can't tell my now, I do not like xml configuration files. I will tolerate a few (like web.xml, jetty's and iBatis') but I won't add any more if I can help it. You can't debug xml files.

Jetty (6) is the only application server I will use or support. It's fast, faster and fastest at serving content, supporting Ajax (via continuations) and staying out of your way. Compared to Jetty, Tomcat is a old cat on its 9th life. Since I have no need for EJBs (version 1, 2, or 3) there is zero point in Weblogic (used it for years since version 4.X) or Websphere (briefly hated it as well).

The Fiche framework sits atop the usual servlet, and is fairly normal in how it handles requests. GET requests are passed to Handlers, POST requests (and also ajax calls) are passed to Actions. The URL scheme is very REST like in design:

/servletname/sitename/pagename[/pagedetails]

Pages may be a single page like home or be a type with the rest of the url as the specification, like article/some_nice_title. Pages may collect content (like the home page) or simply show a single content item (like the article pages). Comments are handled as children of content items. Versioning support means each modification of content can be rolled back. In the near future I will be able to stage changes to pages and content so I can preview the site before making it live.

In the second installment I will talk about AFL and the features I plan to support.

My Tags:

  • x: Feb 20, 2007 10:58

    x

  • y: Feb 23, 2007 08:03

    y

  • Add Comment

Name:


Optional URL:


Comment:


Save Cancel

Copyright © 2007 By Andrew Wulf