You know how it is, idly chatting in #scribus, compiling a little, surfing here and there… and suddenly you want to know the distance from Montreal (site of LGM2 ) to Victoria (site of FOSS4G 2007 ). Nothing as easy as that, just ask Google Maps for directions and it’ll give you the km number for free. Unfortunately there are quite a lot of Victorias around the world, so you have to choose the correct one (in British Columbia). But seeing other locations listed, from Texas to Australia, you obviously try some more, just for the fun of it (maps.google.com sadly only offers destinations on the North-American continent). Ok, no route to Victoria, Australia, nor to Victora, Seychellen — not really surprising.
The distance to Victoria, Texas is even shorter than to Victoria, BC, that’s a (small) surprise. But now: Google Maps Deutschland does have a route to Victoria, UK. Yippee!

google-map-montreal-victoria.png(click to enlarge)

Of course you wonder how to take the longer, middle leg of this journey, but — don’t fear! — Google Maps Deutschland has a solution in its itinerary for that, too. Read for yourself. :-D

/Andreas

P.S. And here’s what we really need: the route from Lyon to Montreal (this time in English language):
http://maps.google.fr/maps?f=d&hl=en&saddr=Lyon&daddr=Montreal

Here is my chance as the primary program administrator to break out the good news. As of today we are in the GSoC 2007 program. Prospective student GSoC developers are encouraged to check out our ideas page and start working on their GSoC plan. See End-to-end Publishing Solution plan for a first example. Of course, every prudent student will make themselves thoroughly acquainted with the Advice for Students GSoC page and proceed according to the guidelines. By the way, if your idea is great, but somehow is not on our ideas page then head straight over to the Blue Sky section of our GSoC ideas page and add your own. Be sure to make your appearance on our mailing list and in IRC #scribus on freenode to make our acquaintance and to start working towards finding a mentor. I (malex) and prokoudine - our backup administrator - will be responsible for organizing the process. So, get going, have fun, and be happy!

Last year I was fortunate enough to be one of the coders COSS hired to code for an open source project. My project of choice was Scribus. Now it’s your turn. COSS is extending the deadline for Kesäkoodi 2007 applications. So if you think you can fill in their requirements submit your application to COSS. If you would like to apply but don’t know what to do you can always contact me at [riku ät scribus dot info] for ideas (my ideas will be Scribus related). I’ll be happy to work as your mentor in a Scribus related project.

From KK2007 FAQ:

WTH is this?

COSS and a number of companies are hiring students to code in open source projects.

And it works actually how?

Here’s it short: The hacker first chooses an intresting open source project in which she / he is willing to contribute in and preferably consults a representative of the project about a suitable programming task. Based on this, the hacker will then create a project for her/himself, makes a project plan and sends an application to Summercode 2007. A panel consisting of representatives of FILOSI universities and institutes will pick the finalists, see criteria. Finalists can correct and polish their plans and make a throughout and short presentation of it for the final, where the representatives of funders will pick the chosen hackers, who’ll be announced shortly after the event. The summer is reserved for hacking and documenting and the fee is paid monthly if the project is advancing as planned.

While we recently released 1.3.3.7 into the wild, one of the tools we used for the first time is the new OpenSuse Build Server. So why the rant ?

In a word it is terrific. :) It still is in “beta” phase and there are features to be added. However, it is a major helper for time stretched developers to package their application in a sane way and then automagically push them to rotating mirrors worldwide without a lot of fuss.

It is all fine and well if you write the best code in the world if no one can use it. And that means packaging. For Linux, that means creating a .deb or creating rpm packages which many Linux distributions use.

The magic is one can upload the source and a spec file which is a recipe to build rpms. Select which platforms to build, push a button and voila. In an hour or two you have all your packages built in a carefully controlled manner. This allowed us to supply 1.3.3.7 for all Suse releases 9.3+ including the Enterprise Desktop 10 for the first time. Even better for users, they can use what ever tool be it Yast, Smart or Yum to add these repositories to get the latest greatest Scribus with all dependencies automatically satisfied. Push button packaging for users too :)

In part two, I’ll explain the master plan and I’ll rant a bit more about why this is all a Good Thing ™

So a big thanks to darix, adrianS and the Build Server team for making such a nice tool and allowing us to play.

We rant and rave about Scribus’ PDF quality and its ability to make commercial grade PDF by mere mortals. A design decision made early on to focus on robust PDF export was in hindsight a brilliant one by Franz. OK, now that is one side of the PDF coin. The other side is parsing and editing PDF. A task even more complex in some cases and in no case trivial.

We often get asked when Scribus will edit PDF and if there are tools which will do it easily and are open source. The first answer is eventually - but it will not be easy, given the team’s collective knowhow about PDF. The second is nothing open source beyond some minor manipulation like splitting, merging and some imposition. And while Acrobat Professional can do some limited PDF editing, it cannot do everything. PDF editing in the way most people perceive is the realm of pre-press specialist Acrobat plugins which are expensive and require some knowhow of pre-press and commercial printing. They also require Acrobat Professional.

Today, I had a chance to get a cvs build of PoDoFo and its GUI browser running which uses cmake and Qt4. PoDoFo is the first serious attempt to write an open source pdf library and tools which can both parse correctly the internals of a PDF and edit PDF. :-)

 

Podofo Browser walking through the internals of a Scribus created PDF

Above is the PoDoFo browser tool. To the untrained eye, you’d say “So what?” To the trained eye what you see is the ability to parse, select and even edit the contents of a PDF. Things like extracting and replacing images. Or eventually re-embedding fonts. Or editing the text within the PDF. Or …
So what it is about PDF that makes it so difficult to edit? Simply parsing PDF correctly is difficult enough. One, it is a non-linear format. You can put the fonts after the text and put the images above them if you want. Content is embedded in what are known as streams and one needs to decipher on the fly the type of stream it is, how it is compressed and where it belongs in the file, along with things like how it is displayed. Oh and objects are reusable too. You cannot simply fire up your preferred editor and start hacking.
PDF was always meant to be “Electronic paper” if you will. In final finished form the idea is it will display the same everywhere no matter what platform or device. (You can display PDF in a Palm Pilot easily.) It was never ever meant to be edited once created.
As I said, not trivial. Then, even if you figure all this out, then you need to reassemble all these bits in a PDF coherent way which won’t break when reloaded in a viewer.

So, PoDoFo is the first real open source PDF editor. Plus, it comes with some other very useful tools for PDF. It will be in the future I hope a PDF library to allow lots of applications to create high quality PDF with the same kind of simplicity and robustness Scribus has now. The key thing is now it is able to get to the much of the low-level bits inside of a PDF. From this lots of magic will come in the future I am sure.

Ready to do some surgery on a PDF ?

One of the oldest feature requests for Scribus have been character styles and hierarchical styles. These are now finally implemented in Scribus 1.3.4cvs. Also Tsoots has done some good work on the new Style Manager, which replaces the old Paragraph Styles dialog, and he will hopefully describe it in a blog entry soon. I’ll take some time here to explain the new concepts behind styles.
Before 1.3.4 there were only named styles for paragraphs and direct fromatting for character styles. Now there are named styles for both paragraph styles and charstyles; and also direct formatting for paragraph styles and charstyles (Yes, this means you finally don’t have to define a new paragraph style if you just want to change the margins for one paragraph).

In the following I’ll describe the logic behind hierarchical styles as it is implemented in Scribus 1.3.4. Paragraph styles, charstyles and line styles behave much the same in this respect. Also other styles which are implemented in the future will share this behaviour. I’ll use paragraph styles as an example in this explanation.

What does “hierarchical” style mean? It means that each style does not just define a bunch of attributes, but also has a flag for each attribute which tells if it is defined in this style or inherited. So consider a style A has as parent a style B:

Style A Style B
parent: “B” parent: “”
font: “Courier” defined font: “Palatino” defined
fontsize: “12″ inherited fontsize: “12″ defined
color: “black” inherited color: “black” defined

If you now format some text with style A, it will take the attributes (”Courier”, 12, black). If you change the fontsize of style B, then style A will automatically pick up the new fontsize. On the other hand, changing the font from “Palatino” to “Times” will not affect style A.
The new stylemanager displays attributes in bold if they are defined locally and in regular font if they are inherited.
It’s also possible to have chains of inheritance, eg. A has parent B, B has parent C, and so on. Only cycles are not allowed.

So what about direct formatting? That is implemented as unnamed styles. I also call those “physical styles” and the named ones “logical styles”.
Physical styles are attached to some content, eg. a run of characters, a paragraph or a story (that’s the combined text of a chain of textframes).
A physical style can have a logical style as a parent and then inherits all attributes which are not set directly.

There’s not only a logical hierarchy but also a physical hierarchy of styles. The physical hierarchy depends on the nesting of elements, so a physical style will pick its attributes from character run, then paragraph, then story, then document. See figure for how those contexts are nested.

Figure: Nesting of contexts

nesting of contexts

So how do I know which attribute value ends up in the style I use? Well, it follows these rules:

1. if the attribute is locally defined (not inherited) in this style, the local value is taken.
2. if the style has a named style as parent, the attribute is searched in this named style(yellow line). If neither the named style nor its ancestors (red lines) define the attribute, go on with 3.
3. the attribute is searched in the next physical style in the hierarchy (follow blue lines). If it’s not defined in any of those styles, go to 4.
4. Use a compiled-in default value.

The documents default style will define all attributes, so rule 4. is never reached.

Ok, that’s all. Complicated? Probably, but I still hope you like it :-)

One of our translators and contributors mentioned to me that they had comments to the effect that Scribus is a “closed shop”. In other words, the team is really not open to adding new developers and contributors. So let’s dispel that rumour right now.

It is not true. Not now, not before, never.

To an outsider, it might appear that way, but not by intent. The Scribus Team is probably a bit different than most in that we are a bit older, perhaps than the average OSS team. Several of us have a few grey hairs on top – but that is a Good Thing for many reasons. :-)

Our experience in various parts of graphics and pre-press has given us a sense of direction about what not only Scribus needs, but what to avoid like faux bold and italic . Sometimes avoiding the same architectural and “feature” errors we’ve suffered as users in the past is just as important as ultra coding foo.

One other thing which is important to understand that we do work very harmoniously as a team. We share the same core values of respecting our users and each other’s unique talents and capabilities. We choose, compliments before criticism, cooperation instead of flaming and division. This, I think, is transmitted through in the way the community surrounding Scribus has grown. Note the lack of flame wars on our mail list. That is something I think we can be proud of and if we are protective of that – so be it.

However, get closer and you will see that Scribus is far from a “closed shop”. Surrounding the core team is a larger circle of folks whose contributions are not only valuable, but necessary. One look at Help > About, the mailing list or a look at the Wiki is evidence enough for me.

To the contrary, we do want more coders, more translators, more packagers and folks to help with testing bugs. Jump in! The water is just fine.

You might know that being an Open Source developer in general, and a Scribus developer in particular, allows you to travel all over the world, meet interesting people and have a nice dinner with them. This time I attended the GNOME Live! summit in Boston at the MIT. Thanks to Behdad Esfabod from Pango and Ed Trager from Unifont it also hosted the first FLOSS Text Layout Summit this year; and that was the reason for me being there (not  to prepare for porting Scribus to GNOME and not  for a tea party :-) ). Also attending where people from Qt, KWord, fontconfig, SuSE, DejaVu fonts, and SIL. So what was this about? You probably all had your share of font related problems, so it should be no surprise we talked about that — a little. The major part was concerned with international text layout. Ed Trager wrote a nice introduction on what this is about. If you are only familiar with Latin script, read in this article all about writing directions and how characters change their shape according to context. International layout usually includes the following steps:

  1. Itemize — find runs of characters which have the same script and font
  2. Shape — apply script specific substitutions and positioning
  3. Find linebreaks — most applications use the simple and straightforward approach, but Scribus will do something more sophisticated soon.
  4. Justify lines — by extending spaces, scaling glyphs, substituting glyphs or by script specific means, e.g. Kashida in Arabic.
  5. Render glyphs — that’s easy once you have all glyphs with their exact position and size.

Currently there exist three different libraries for international text layout (Pango, Qt and ICU) which all have their pros and cons. At the text layout summit we found out that we all share the common vision of a unified library. Why is that? Well, first of all, writing such a library is hard. Also you need a lot of people to check the results for each existing script. Second, there are still many scripts which are not supported yet, and the hope is to support scripts faster if we combine our efforts. Finally, users don’t appreciate if their native language works in some applications but not in others, just because they use different libraries. You can see this within Scribus if you compare for example Arabic text in StoryEditor and on canvas (The SE uses Qt and displays Arabic correctly).
But we also reached some decisions. As a first step we will write a unified shaping engine. I was surprised how fast the Qt and Pango developers agreed on a common datastructure for such a library (well, that’s only tentative, but I expected some long discussions there). Trolltech said they would consider relicensing their shaping code in Qt so it could be included in a common library. Finally we decided to develop the unified shaping engine on top of HarfBuzz. HarfBuzz is a library which handles OpenType tables and is already used by both Qt and Pango. The unified shaping engine will also be called HarfBuzz and will initially support OpenType fonts. Legacy Type1 and TTF fonts can be used for basic shaping in Latin script and for Arabic if they contain the necessary Arabic presentation forms. Later we also want to integrate Graphite, a new open source font technology from SIL, and platform specific glue code for AAT (Macintosh) and UniScribe (Windows).

Ah yes, in the future Scribus will use this unified shaping engine directly. KWord decided on using Qt, and most other applications will also use either Qt or Pango. There was also some interest from FireFox, OpenOffice, Inkscape and Abiword developers, but so far I can’t tell if they would use the new unified shaper directly or not.

Well, what else? We have a combined planning/agenda/minutes webpage. There are also links to some slides and audio recordings of most talks.

The organisation of the summit was not as good as LGM, though. Ok, there was free lunch (American pizza for all), but we had trouble to use a projector, had to organize audio recording ourselves, could only find (near dry) green and yellow pens for the whiteboard and worst of all: no coffee! At first I hoped I had discovered a very geeky coffee machine in one corner of the MIT media lab, but that was just a liquid nitrogen tank. :-(

But all in all, very nice people, very geeky setting, very good dining, and interesting and productive talks. And I had ExScite as my private sightseeing guide on my last day, thanks ExScite! :-)

This afternoon and tonight, I had a chance to thoroughly beat up on a very recent snapshot Krita 1.6svn :)

What I wanted to see was if I could at last do simple RGB to CMYK conversions and

  • Avoid diving to the command line.
  • Would be usable and understandable for mere mortals who are not color management geeks.
  • Do this in simple 1,2,3 steps that could be explained to Scribus users for reliable transforms from RGB into CMYK.
  • Display reliably in Scribus what would end up coming out of commercial printer.

Krtia succeeded with flying colors.

Now, with Krita 1.6 installed you can take any decent RGB jpeg, png, PSD or tiff open it and convert it to CMYK and then safely, reliably save it as a CMYK TIFF which then imports into Scribus without a fuss. My first tests show that the images in the PDF’s exported look exactly like the original RGB jpeg and PSD files I opened in Krita. As close I get when doing the same in PS6.
The official announcement awaits, but it should be out and available packages soon.

I’ll rant more later about the details ;)
This is a huge step for open source graphics. Huge.

It’s a common misconception that a font is a bunch of characters. That’s not so. Fonts (or, as Adobe likes to call them, font programs) contain glyphs. Glyphs are the shapes which are painted to screen or on paper. Characters are more abstract and convey some meaning, eg. the latin character ‘A’ or the diigit ‘7′. Even a single font can have more than one glyph for those characters, eg. a swash ‘A’ or an oldstyle numeral ‘7′.

Until now Scribus had the same sloppyness regarding characters and glyphs. Internally Scribus stores Unicode characters and for output it used to map each character to exactly one glyph. I’ve started to change that, though. Here are some use cases where it might be necessary to do something else:

  • using different glyphs depending on style, eg. SmallCaps or OldStyleNumerals
  • mapping the pagenumber placeholder to several digits
  • smart hyphens don’t have an associated character stored with them
  • ligatures

So here’s the new scheme:

Scribus still stores a sequence of Unicode characters. For output, two more sequences are generated: a sequence of items and a sequence of glyphs. The items provide a glue between the unicode chars and the glyphs. The smallest possible item is a cluster, that is a character and all its marks, e.g. ‘a’ + ‘^’. This would be connected to the glyph ‘â’. The other way is also possible: the unicode ‘â’ is mapped to two glyphs ‘a’ and ‘^’ which are positioned in just the right way. Usually an item will correspond to a syllable, though. The details of items are only relevant to Scribus’s layout engine, so I won’t go into that here. Just keep in mind that items are used to break the 1:1 correspondence between characters (what you type) and glyphs (what you see).

The item and glyph sequences are created during layout. At the same time the exact position and size of each glyph is calculated. A sideeffect of this is that Scribus now uses the same data for screen, PS and PDF output (before that the PS and PDF output routines did part of the layout themselves — bad idea).

The current code is still limited. Currently it’s only used for hyphens and pagenumbers. The next step will be to use the new NLS data structures and take advantage of OpenType features. So stay tuned: true smallcaps, ligatures and oldstyle figures are on their way…

« Previous PageNext Page »