Update: Good news! Jaroslav Toulash emailed me that he published a book on Practical API Design !!!
Looks like Brian McAllister may be preparing a talk on Designing Elegant APIs.
I've been very interested in good API design for a long time. But I could never find a single book on the subject. Many design and programming books provide good advice and guidelines that are essential for designing good APIs, but none of them tackles the matter directly.
I reviewed "Interface Oriented Programming" a few months back with disappointment. It may not be a bad book, but I felt it was quite basic and superficial. May be my expectations were too high, and too focused on API design.
Over time, I collected some links on the subject and shared some with Brian:
API: Design Matters - Article by Michi Henning, ZeroC, for ACM Queue magazine.
Interface Design, Best Practices in Object-Oriented API Design in Java, by Bill Venners - This one is a Book in progress!. Unfortunately, the last updates seem to be from 2002 or so. I'm not sure if the project is still alive. Anyway, it contains many good articles.
How To Design a (module) API - A great page on good design practices for writing APIs. It's tailored for NetBeans module writers, but is still pretty general and gives very valuable guidelines.
How to Write APIs That Will Stand the Test of Time - A session I attended at JavaOne 2006. Presented by Tim Boudreau and Jaroslav Tulach, members of the NetBeans team. It focuses on API evolution and compatibility, but also covers usability and other guidelines. I guess they are the main guys behind NetBeans' How To Design a (module) API page linked above.
I had the chance to speak with Jaroslav after his session and suggested he should write a book on this topic. I emailed him the references I had, and he said he would try to draft something by the end of year.
API Usability - Focused on making APIs easy to use. The article applies general usability principles (commonly used in GUI design) to the design of APIs.
Measuring API Usability - An interesting article by a usability engineer at Microsoft, published on Dr. Dobb's. It favors the use of scenario-based design to achieve usable APIs, and explains the use of their "cognitive dimensions" framework for measuring API usability. He also blogs about API usability.
Java API Design Guidelines - A good article from a developer who was also surprised by the lack of a book about API design. He collected some advice and additional links.
XOM Design Principles - Some design "principles" followed by XOM (an XML parsing library).
The Most Important Design Guideline? - A short article by C++ guru Scott Meyers.
Humane Interface, Minimal Interface, Designed Inheritance, DSL Boundary, Duck Interface, Fluent Interface, Public vs. Published Interfaces - Some of the many great writings from Martin Fowler. These selection is on specific topics that are relevant to API design. Most of them are quite recent.
Programmers are People, Too - An article by Ken Arnold for ACM Queue magazine: "Programming language and API designers can learn a lot from the field of human-factors design."
So, let's hope Brian gives his talk at a big conference, signs a contract with a big publisher and fills the void.
Most software products and logging frameworks produce entries in their log files with different severity levels (say, from Debug to Severe). Then, you can configure which level of messages you want the application to log.
At development time, you may want all messages to be logged. But once in production, you normally configure things so that only Warning and higher messages get logged (for performance, space and other reasons).
The problem is that when a Severe message arises you may need more detailed (Debug level) information in order to understand the actual cause of the problem. So, a common practice is to increase the logging level at that point, and wait for the problem to happen again –not very effective.
This is one of the problems that motivated the guys behind LogBag. The idea they propose is nice and simple:
Make the system write only Warning (and higher) messages to the log, but when such a message is logged, it should also include some lower-severity messages that occurred right before and after this one.
It’s one of those good ideas that might seem obvious, but for some reason nobody thought (or did anything) about before.
To implement this, the logging system could keep a buffer of the last N messages generated by the application, and when a Warning (or above) message comes in, the whole buffer is flushed to disk. This should not be too hard to implement, since most logging frameworks already use some kind of buffering, in a producer-consumer pattern, to improve concurrency and performance.
From the LogBag site:
You keep on putting things in a Log bag. Then whether the entire log bag gets written or not depends on the highest severity level in the entire bag. So either you write everything or you write nothing.
I think it’s a nice idea. I bet the technical support guys would like it too :-).
I just released a new version of AntDoclet.
No major changes. Now the comments of inherited methods are properly extracted from the parent class comments.
Thanks to Daniel Lindner for this fix.
Tom Baeyens (from jBPM) wrote an interesting article about the over-promises of some BPMS (Business Process Management System) vendors.
His dicussion addresses two points:
BPM vendors often overpromise by hinting that BPM tools can unify analysis with implementation
Lack of integration between processes and plain general purpose programming language
From my experience with Fuego (now BEA AquaLogic BPM) it’s safe to say that it doesn’t fall in that category. I don’t know what vendors he’s referring to because, honestly, I’ve never spent serious time trying or analyzing other BPM vendor products.
On point #1: BPM over-promises
Tom is on the camp that any serious BPM development (or any “software development” for that matter) will always require a developer. He believes that the case where a Business Analyst defines and deploys a business process without help from developers is limited to BPM vendor demos.
Keith Swenson, on the other side, replied suggesting that BPM could be like Spreadsheets: many business people now use Excel to build their own software solutions without help from developers.
Both arguments have their merits, and I think reality is somewhere in between.
From my experience building BPM solutions for customers, I believe that the ideal of complete IT independence is not feasible today, nor in the near future, except for very simple cases. (not only because of technical, but also cultural reasons).
I’d question that the Spreadsheet example, while interesting, is quite simplistic. What a business analyst can build with a spreadsheet is a lot simpler that what you’d normally need a BPM for. For instance, a spreadsheet “application”:
- is a single-user solution
- Has no concurrency
- No security requirements
- No integration with other system (or minimal integration with a database)
- No transactionality
- Does not “expose” functionality to other systems
- No “enterprise” deployment (just run it in your desktop)
Obviously, you can build a single-user, non-transactional, non-integrated solution with a BPM tool, and yes, a non-developer could do it. But you’d be falling into the toy-example case that Tom hates about BPM demos.
But, I do believe that with a good BPM tool the level of abstraction can be raised so that less technical skills are required to build usable solutions. And, over time, that level will be raised more and more. I’ve seen business people that, after being involved in a couple of BPM projects, they do more and more by themselves, to the point of building robust usable prototypes on their own.
On point #2: Process and general programming languages
With point #2 of Tom’s original post, he states that it’s critical to integrate the process model with a general purpose programming language.
I feel comfortable with that idea, because that’s how it’s been done in FuegoBPM (ALBPM) since day one. And although I cannot say it is the best approach (since, again, I have no real experience with other BPM tools), I can testify that it works pretty well in practice.
As a side note, we believe that it is bad practice to overload the graphical representation with massive amount of decorations to try and express every little technical detail of the process. But the most important point is that you can create a common language between the technical developer, who is responsible for making the process executable and the business analyst, who is responsible for communicating the requirements.
I agree with this, and it’s important. The graphical process design should be at a high-level of abstraction, in the business domain terms, so that anybody can understand and follow the business process flow.
I think the value is not on the fact that a Business Analyst can design an implement a solution on his own. Instead, the value is that the Business Analyst can build initial designs and prototypes, and then keep participating on the process design during the whole development lifecycle, because while developers implement the low-level details, the graphical model (if done right) is still understandable by everyone. And, this graphical model is not just documentation, it’s the real thing, executable.
As a side-effect, BPM is a great tool for facilitating iterative development, helping to close the communication gap between the business owners/users and developers, allowing early and continuous feedback into the development process.
A great interview with Amazon’s CTO Werner Vogels appeared in ACM’s Queue magazine. A must read if you care about how to run a modern IT organization.
The SOA acronym has been abused to the point of becoming a dirty buzzword for many. The interview shows how Amazon applies service-orientation concepts with a very pragmatic approach.
WV: …The services model has been a key enabler in creating teams that can innovate quickly with a strong customer focus. Each service has a team associated with it, and that team is completely responsible for the service from scoping out the functionality, to architecting it, to building it, and operating it.
Excellent! That’s what I ment with “Project-Oriented-IT”. I’m glad Amazon has been doing that successfully.
Here’s why it works:
WV: …Giving developers operational responsibilities has greatly enhanced the quality of the services, both from a customer and a technology point of view. The traditional model is that you take your software to the wall that separates development and operations, and throw it over and then forget about it. Not at Amazon. You build it, you run it. This brings developers into contact with the day-to-day operation of their software. It also brings them into day-to-day contact with the customer. This customer feedback loop is essential for improving the quality of the service.
He obviously explained it much better than I did, and with real-world experience to support it.
It shows that the cultural and organizational aspect of a SOA initiative is more important that the technologies used. SOA is not just about building “software architectures”. If done right, it is a better way of organizing IT operations.
You don’t even need to standardize on specific technologies/tools to run an organization with a service model:
WV: I think part of the chaotic nature the emerging nature of Amazon’s platform is that there are many tools available, and we try not to impose too many constraints on our engineers. […] Developers are like artists; they produce their best work if they have the freedom to do so, but they need good tools. As a result of this principle, we have many support tools that are of a self-help nature. The support environment around the service development should never get in the way of the development itself.
Don’t force tools down the developers throat. (Any manager reading?)