June 2006

It is not uncommon that when a program like Scribus grows so quickly and gets so many features added in such a short amount of time that some cleaning is necessary now and then. Throughout the 1.3.x series we have cleaned a lot of code up, fixed a lot of bugs uncovered by the cleaning, as well as continuing to improve the whole internal view of things.

So far in the 1.3.4cvs timeframe I have not had a lot of spare time as I started a new job (yay!) so the major rewrites of the 1.3.0->1.3.2 that I was able to do before were put on the backburner. In this time Andreas has been going nuts with the text system rewrite.. amazing stuff is happening there. So, in my spare time I’ve been testing, discussing etc but that gets a little boring after awhile, so this week I decided to remove one of the bugbears I’ve been hanging to do for a long time. That is, remove the global extern ScribusMainWindow* ScMW and then fix up all the resulting mess. This was used for easy access to the main window and its code in a previous lifetime where a lot of the main document code was held within the main window itself as well as a general easy (read nasty) way of finding your way back to the main application.

So, having just committed updates to about 90% of the Scribus files in 1.3.4cvs :), what does this aim to achieve? Primarily, moving the bulk of access down a layer into the document. Theres still a lot of leftover code that now just goes doc->scMW() however once this is eliminated we can start to run more operations without a GUI (read command line) or with multiple GUIs (read SDI mode). Hopefully I removed all the uninitiated pointers around and added ones where necessary :). Still works ok here, but we never said the changes in 1.3.4cvs were not going to get a little bumpy at times :). Heres to a big change, that took a week of spare hours at nights. On ya guys.. 1.3.4 will be a major step forward with all the cool transparency stuff and text and the to-come style manager from Riku and.. and.. and…

Well, that might depend on how you do it. Using DOM is quite easy, although it might be a bit tedious if you have to traverse the whole DOM tree to create your application specific data structure (and of course it’s slow). Using SAX is faster, but programming all those handlers can be just as tedious. In any case you’ll likely have a lot of code which looks like this:

if (tagname == “SCRIBUSXML”) {
   doc = new ScribusDoc();
else if (tagname == “PAGE”) {
   doc.addPage(new Page());
else if (tagname == “STYLE”) {

Not nice at all. Of course Java developers will smirk now and say “that’s no problem for us; we can use Jakarta Commons Digester.” The curious C++ programmer might check out their webpage or the introductory article on JavaWorld and get envious. Really envious.

An example program with Digester might look like this:

Digester digester = new Digester();
digester.setValidating( false );

digester.addObjectCreate(“SCRIBUSXML”, ScribusDoc.class);

digester.addObjectCreate(“STYLE”, ParagraphStyle.class);
digester.addSetNext(“STYLE”, “addStyle”);

digester.addObjectCreate(“PAGE”, Page.class);
digester.addSetNext(“PAGE”, “addPage”);
digester.addSetProperties(“PAGE”, “NUM”, “number”);

ScribusDoc result =
   (ScribusDoc) digester.parse( “example.sla” );

So instead of programming all those tedious if-cases, you just add rules which tell
Digester what element tags and attribute names to look for and what to do if found: create new objects, call setter methods, or do some other stuff. Then you call the parse method and at the correct times the digester uses your rules to create your datastructure. At the end you just query the root object and there you go.

Unfortunately, my current pet projects are hooked on C++, so I tried to achieve a similar solution with C++ templates. This is what I came up with:

Digester digester = new Digester();

digester.addRule(“SCRIBUSXML”, Factory<ScribusDoc>());

digester.addRule(“STYLE”, Factory<ParagraphStyle>());
digester.addRule(“STYLE”, Setter( & ScribusDoc::addStyle ));

digester.addRule(“PAGE”, Factory<Page>());
digester.addRule(“PAGE”, Setter( & ScribusDoc::addPage ));
    SetAttribute( & Page::setNumber, “NUM” ));

ScribusDoc result = digester.result<ScribusDoc>();

Do you like it? Me too. 🙂
The initial version of the library is available in current CVS in the “desaxe” directory. I’ll add some documentation and examples soon.

Some internals:
The deSAXe library uses a SAX parser which reads the XML document and creates events each time a start tag, end tag or a string of characters is read.
Setter/Factory/SetAttribute are examples of Actions. When a rule fires, the SAX events are forwarded to the corresponding Action. If more than one rule fires, all Actions reveive the events in the order they were added to the digester. Internally deSAXe uses a finite automaton to keep track on which rules are active, so it’s pretty fast (unlike Jakarta Commons Digester 🙂 ).

Looking forward to your comments (please use the mailing list since atm comments are disabled here).


P.S. Yes, I do enjoy parsing XML, now! 🙂
P.P.S. Apart from the pun on SAX the library is named after the coolest Scribus user in Australia, Elaine de Saxe. 🙂 Hi Elaine!

Finally, loading of 0.x, 1.x, 1.2.x and 1.3.x=>1.3.3 file formats is now coded with the FileLoader plugin. Work can begin fairly quickly on the file format code for 1.3.4, OR at least which was my main urge to get this code done: massive format changes for Andreas and Riku to support inherited text styles, Unicode reading and writing instead of control characters, etc etc. This will mean that we will no longer need to safely change 1.3.3 loading code.. its handled separately now. To do: File saving of 1.3 docs via the plugins, and.. if anyone got adventurous, saving back to 1.2 format.

Update: Saving of docs to 1.3.x format is done.
Update: Created a new 1.3.4 plugin for loading and saving. This is now the default plugin used.

One thing about GIMP which generates lots of virtual ink, but not much enlightenment is the lack of CMYK support in GIMP. Post a subject on slashdot with OSS + graphics and watch the flames begin… About 50% of the Scribus 1.0 story was GIMP vs PS flaming… Sad.
Many many people have moaned, complained and sworn off GIMP for its lack of CMYK and/or other perceived deficiencies, yet the number of people to step up and put the effort to make it a reality seems to me awfully small.
In hindsight, both working on Scribus for more than 5 years now and in pre-press longer has led me to the opinion that adding CMYK to GIMP prematurely would have been a disaster.
My observations:

  • In defense of the GIMP developers, I too would be really sick and tired of people coming on IRC, or various mailings and asking/demanding/complaining about this feature or that feature. So, if they are less than polite when you are person number 998 to ask/whine/demand/plead for the same thing that is entirely understandable. You are at fault for simply failing to google the question. The answers are there…
  • Unlike 99.95% of the people who have used, flamed or discussed GIMP, thanks to LGM, I have actually met a fair number of the GIMP developers face to face. They are nice people and they did a terrific job organizing LGM. We could not have been more welcome.
  • Trust me, they know CMYK support is a long standing request. They are neither blind, nor idiots and perfectly well understand why it is so sought by end users.
  • Those who have not worked on a project like Scribus or GIMP or Inkscape may not be aware how difficult it is to make CMYK work properly. We are rightfully proud of the CMYK and color management capabilities in Scribus. But it is not perfect and we still have more work to do there. Thanks Marti.
  • I think they are not only correct, but completely within their rights to say they do not wish to make a Photoshop clone. You know what? Photoshop, while it wonderful for some things it is far from perfect…
  • I use both, side by side and they both have their quirks and oddities, There are things GIMP just does better, more efficiently than PS. GIMP’s ability with retouching old photographs is extremely good and its handling of PNG is superior in my experience. The new screen grabber in 2.3.x rocks.
  • Users running GIMP on Windows complain about all the palettes and the lack of SDI (Single Document Interface.) While I don’t like MDI myself, the GIMP developers are correct: Windows has a sucky window manager. What is also interesting to note is that PS 6 and 7 run perfectly well on recent versions of Wine — thanks to Codeweavers. Guess what? All those PS palettes are not so nice under KDE or Blackbox on Linux 😉 A perfect example of an application working in an environment where it was not really designed for. A note for all future or potential reviewers of GIMP, please test it on Linux or some Unix with a decent window manager, it does make a difference.
  • Those who complain how it runs on OSX or its lack of native port should: 1) Complain to Apple to make their X-11 work more seamlessly with Aqua. 2) Give or loan a pile of well equipped OSX boxes to the GIMP team and while you are it, start or pay a team of experienced developers to port GTK2 to OSX.
  • Both PS and GIMP are complex applications that take time to master. For longtime Photoshop users to try GIMP for a short time then reject it out of hand are just as ignorant of the GIMP fanboys who blindly recommend GIMP as a replacement for PS. Sorry, it is not. The GIMP developers have no shame in that being so… You have to “Think Differently”, to borrow a well known tagline 😉
  • CMYK and spot colors by themselves are not patent encumberd. They are actually part of the open published standards for Postscript and PDF. Anyone saying anything different is clueless or spreading FUD and/or openly demonstrating their ignorance of the fact.

So it might be useful to ponder a moment, why Scribus was able to integrate CMYK and color management seemingly so easy? A good question and the answers are hopefully instructive. The most important? You need the correct development environment that can support adding and maintaining such a feature, the lack of which is not the fault of the GIMP developers:

  • Unlike GIMP, CMYK was from day one a necessity for Scribus. Scribus without CMYK and spot color support equals Scribus being a less than useful toy. Adding useful color management makes that so much easier. Having littlecms as a drop in library to handle all the CMS chores is truly a gift. GIMP and Inkscape are truly useful applications of their own, right now, for a huge audience without these features.
  • You need developers who have pre-press and print design know how — very different from RGB/Web/Video image editing. Jean on our team probably does have Video/Web imaging foo (he’s a pretty skilled imaging engineer, but I doubt the rest of us would be a good team to write an image editing app or something like GIMP GAP.)
  • You need end users who understand how to use these features properly, willing to dedicate time and effort to test and guide the developers where they are right and wrong. Thank you Louis, Christoph, Maciej, Johannes and ExScite among others for relentlessly thrashing CVS.
  • You need a testing environment which is capable of testing CMYK output in a realistic fashion. We were blessed early in Scribus’s life to have clients who allowed me in early days to run and test Scribus in a real world pre-press environment where you could compare Scribus’s and other application’s output side by side. Bring heaps of money to recreate that from scratch. 😉
  • Think about spending tens of thousands in special pre-press software and special equipment. Then, a bunch more time to learn how to configure and calibrate these bits properly. It took the better part of year studying and testing before rolling out my first real color managed workflow. Even then it takes time to keep up with the latest CMS development and standards.
  • The current GIMP developers inherited a code base with 8 bit color. Implementing 16bit and larger color spaces on that code base is not trivial. For some parts of the GIMP, I would imagine there are parts where a complete rewrite is in order. The efforts they put into improving 2.0.x and 2.2.x were far more valuable for a very wide part of their user base. Stability and hundreds of other upgrades small and large benefit all users. I cannot remember the last GIMP crash in 2.2.x.

So, now before ranting about the lack of CMYK or spot color support in GIMP, maybe you might have a more realistic understanding of the undertaking. I’m perfectly content with GIMP now and I only expect it will be better in the future. The revitalization of GEGL is a big step. Maybe, GIMP 2.6 or 3.0 will have CMYK and spot colors. I expect when it is added it will work pretty well right out of the box. That’s worth waiting for…

Yesterday was the first day of Kesäkoodi 2006 which allows me to drop off my regular (sometimes boring) summer time job as a train driver and lets me concentrate fully on Scribus. Thanks to HKL they made it possible for me to keep my train driving license (so I’ll be still able to work there next summer if nothing else will be found) and accepted the three months off I was asking. Whether or not I’ll be driving metro next summer remains to be seen but I hope to find something that involves a bit more coding then too.

I applied to Kesäkoodi 2006 with a project that includes a new get text api. This will be needed because the work Andreas is doing with the new layout system will break the old one. It also gives me a possibility to improve the system. My plan is to extend the current api to include two types of plugins, file format plugins and text/style plugins. After these core modules are done I’ll be also implementing three plugins during the three months time I’ll have (and later more I hope). These three plugins include an improved ODT importer, RTF importer and text and style filter plugin. I’ve added a Kesäkoodi metabug to the bug tracker where it’s possible to read the outline of the project and see my progress as closed bugs.

Few other Scribus related things I’ll be doing during the summer along with the KK2006 include a new Style editor and a new layout for our palettes (at least for the Properties Palette). I’ve also started to translate Inkscape into Finnish and hope to finish it during the three months I’m paid to code Scribus. It’s our sister project after all.

First day went by me cleaning up the office. Must say it always feels larger project than it really is. In a clean office I’ll be definitely more productive. That’s for sure.

After much wailing, gnashing of teeth and general procrastination by many.. today’s show is brought to you by the letters w, o, r, k, i, n and g! The aging (read, since 1.3.1) FileLoader plugin system can finally load a Scribus 1.2.x document without the AGES old scribusXml.cpp anymore. All very technical for those that just want to do publishing, however what it does mean is that once the work is complete, we can bring you a cleaner loading and saving system. Fewer, or preferably no modifications into old loader code for old format docs when adding new features to the new formats. Most importantly, this means that finally we can begin the 1.3.4+ format to handle the new text features. I think that can start next week.