On API Design Guidelines 7

Posted by F 19/11/2006 at 22h31

Update: Good news! Jaroslav Toulash emailed me that he published a book on Practical API Design !!!

Looks like Brian McAllister may be preparing a talk on Designing Elegant APIs.

I've been very interested in good API design for a long time. But I could never find a single book on the subject. Many design and programming books provide good advice and guidelines that are essential for designing good APIs, but none of them tackles the matter directly.

I reviewed "Interface Oriented Programming" a few months back with disappointment. It may not be a bad book, but I felt it was quite basic and superficial. May be my expectations were too high, and too focused on API design.

Over time, I collected some links on the subject and shared some with Brian:

So, let's hope Brian gives his talk at a big conference, signs a contract with a big publisher and fills the void.

Variable Severity Logging 1

Posted by F 24/09/2006 at 19h21

Most software products and logging frameworks produce entries in their log files with different severity levels (say, from Debug to Severe). Then, you can configure which level of messages you want the application to log.

At development time, you may want all messages to be logged. But once in production, you normally configure things so that only Warning and higher messages get logged (for performance, space and other reasons).

The problem is that when a Severe message arises you may need more detailed (Debug level) information in order to understand the actual cause of the problem. So, a common practice is to increase the logging level at that point, and wait for the problem to happen again –not very effective.

This is one of the problems that motivated the guys behind LogBag. The idea they propose is nice and simple:

Make the system write only Warning (and higher) messages to the log, but when such a message is logged, it should also include some lower-severity messages that occurred right before and after this one.

It’s one of those good ideas that might seem obvious, but for some reason nobody thought (or did anything) about before.

To implement this, the logging system could keep a buffer of the last N messages generated by the application, and when a Warning (or above) message comes in, the whole buffer is flushed to disk. This should not be too hard to implement, since most logging frameworks already use some kind of buffering, in a producer-consumer pattern, to improve concurrency and performance.

From the LogBag site:

You keep on putting things in a Log bag. Then whether the entire log bag gets written or not depends on the highest severity level in the entire bag. So either you write everything or you write nothing.

I think it’s a nice idea. I bet the technical support guys would like it too :-).

AntDoclet 1.1

Posted by F 22/07/2006 at 16h43

I just released a new version of AntDoclet.

No major changes. Now the comments of inherited methods are properly extracted from the parent class comments.

Thanks to Daniel Lindner for this fix.

On What BPM Promises 1

Posted by F 11/07/2006 at 13h09

Tom Baeyens (from jBPM) wrote an interesting article about the over-promises of some BPMS (Business Process Management System) vendors.

His dicussion addresses two points:

  1. BPM vendors often overpromise by hinting that BPM tools can unify analysis with implementation

  2. Lack of integration between processes and plain general purpose programming language

From my experience with Fuego (now BEA AquaLogic BPM) it’s safe to say that it doesn’t fall in that category. I don’t know what vendors he’s referring to because, honestly, I’ve never spent serious time trying or analyzing other BPM vendor products.

On point #1: BPM over-promises

Tom is on the camp that any serious BPM development (or any “software development” for that matter) will always require a developer. He believes that the case where a Business Analyst defines and deploys a business process without help from developers is limited to BPM vendor demos.

Keith Swenson, on the other side, replied suggesting that BPM could be like Spreadsheets: many business people now use Excel to build their own software solutions without help from developers.

Both arguments have their merits, and I think reality is somewhere in between.

From my experience building BPM solutions for customers, I believe that the ideal of complete IT independence is not feasible today, nor in the near future, except for very simple cases. (not only because of technical, but also cultural reasons).

I’d question that the Spreadsheet example, while interesting, is quite simplistic. What a business analyst can build with a spreadsheet is a lot simpler that what you’d normally need a BPM for. For instance, a spreadsheet “application”:

  • is a single-user solution
  • Has no concurrency
  • No security requirements
  • No integration with other system (or minimal integration with a database)
  • No transactionality
  • Does not “expose” functionality to other systems
  • No “enterprise” deployment (just run it in your desktop)

Obviously, you can build a single-user, non-transactional, non-integrated solution with a BPM tool, and yes, a non-developer could do it. But you’d be falling into the toy-example case that Tom hates about BPM demos.

But, I do believe that with a good BPM tool the level of abstraction can be raised so that less technical skills are required to build usable solutions. And, over time, that level will be raised more and more. I’ve seen business people that, after being involved in a couple of BPM projects, they do more and more by themselves, to the point of building robust usable prototypes on their own.

On point #2: Process and general programming languages

With point #2 of Tom’s original post, he states that it’s critical to integrate the process model with a general purpose programming language.

I feel comfortable with that idea, because that’s how it’s been done in FuegoBPM (ALBPM) since day one. And although I cannot say it is the best approach (since, again, I have no real experience with other BPM tools), I can testify that it works pretty well in practice.

As a side note, we believe that it is bad practice to overload the graphical representation with massive amount of decorations to try and express every little technical detail of the process. But the most important point is that you can create a common language between the technical developer, who is responsible for making the process executable and the business analyst, who is responsible for communicating the requirements.

I agree with this, and it’s important. The graphical process design should be at a high-level of abstraction, in the business domain terms, so that anybody can understand and follow the business process flow.

I think the value is not on the fact that a Business Analyst can design an implement a solution on his own. Instead, the value is that the Business Analyst can build initial designs and prototypes, and then keep participating on the process design during the whole development lifecycle, because while developers implement the low-level details, the graphical model (if done right) is still understandable by everyone. And, this graphical model is not just documentation, it’s the real thing, executable.

As a side-effect, BPM is a great tool for facilitating iterative development, helping to close the communication gap between the business owners/users and developers, allowing early and continuous feedback into the development process.

Real-world SOA at Amazon

Posted by F 08/06/2006 at 20h41

A great interview with Amazon’s CTO Werner Vogels appeared in ACM’s Queue magazine. A must read if you care about how to run a modern IT organization.

The SOA acronym has been abused to the point of becoming a dirty buzzword for many. The interview shows how Amazon applies service-orientation concepts with a very pragmatic approach.

WV: …The services model has been a key enabler in creating teams that can innovate quickly with a strong customer focus. Each service has a team associated with it, and that team is completely responsible for the service from scoping out the functionality, to architecting it, to building it, and operating it.

Excellent! That’s what I ment with “Project-Oriented-IT”. I’m glad Amazon has been doing that successfully.

Here’s why it works:

WV: …Giving developers operational responsibilities has greatly enhanced the quality of the services, both from a customer and a technology point of view. The traditional model is that you take your software to the wall that separates development and operations, and throw it over and then forget about it. Not at Amazon. You build it, you run it. This brings developers into contact with the day-to-day operation of their software. It also brings them into day-to-day contact with the customer. This customer feedback loop is essential for improving the quality of the service.

He obviously explained it much better than I did, and with real-world experience to support it.

It shows that the cultural and organizational aspect of a SOA initiative is more important that the technologies used. SOA is not just about building “software architectures”. If done right, it is a better way of organizing IT operations.

You don’t even need to standardize on specific technologies/tools to run an organization with a service model:

WV: I think part of the chaotic nature the emerging nature of Amazon’s platform is that there are many tools available, and we try not to impose too many constraints on our engineers. […] Developers are like artists; they produce their best work if they have the freedom to do so, but they need good tools. As a result of this principle, we have many support tools that are of a self-help nature. The support environment around the service development should never get in the way of the development itself.

Don’t force tools down the developers throat. (Any manager reading?)