The L&L Blog
Coding XML Formats in Cocoa
I’ve long been intending to add some technical, coding-related content to this blog. I’ve always admired the blogs of other developers who share some of the coding problems they have faced and solutions they have reached, and in developing Scrivener there are a number of issues that I’ve come across and found solutions for that I’m sure could be useful to other developers. So, this is the first more techie, coding-orientated post to make it to the blog. Those with no interest in Cocoa development, look away now. I have recently been rewriting Scrivener’s file format. As you may or may not know, .scriv files are packages – essentially folders that on OS X just look like, and are treated the same as, files. Were you to move a .scriv file to a different platform, it would appear as a regular folder. On OS X, you can ctrl-click on a .scriv file in the Finder and select “Show Package Contents”. File packages are great for programs such as Scrivener. Regular files usually have to be loaded into the program’s memory in their entirety when loaded; with a file package, the program can just look inside it and open whatever it needs as and when it needs it. Given that Scrivener can import movie, sound, PDF and image files, you can imagine how much memory might get eaten up if it had to load everything into memory right from the get-go. Instead, with its package format, it can just load the binder structure file and then load up each document as you select it in the binder, and flush from memory any large files that aren’t currently being used. For Scrivener 2.0, .scriv files will still be file packages, but I’ve been doing some work on the format of the files inside the .scriv file. Currently all text files are saved internally as RTFD files (RTFD stands for “rich text format directory”). RTF files are a standard rich text format that can be opened on all major platforms (they are essentially plain text files with formatting mark-up), and were designed by Microsoft; RTF files support pretty much everything that Word documents do. RTFD is Apple’s extension of this format. An RTFD file is a file package in itself (as with .scriv files, you can ctrl-click on them in the Finder and select “Show Package Contents” – you will find a TXT.rtf file inside there, for instance, which holds the actual text). Apple designed the format so that such files could also hold QuickTime files and any other file type; but the trouble is that RTFD files can only be opened on Macs. Most of the other files inside .scriv packages use the Apple .plist format – I may have given some of them the .xml extension (.plist files are technically XML files), but internally they are just Apple .plists. (.plist stands for “property list”). This format for Scrivener files was generally a great and solid 1.x file format. It works, and it doesn’t take too much code to maintain on my part – the Cocoa frameworks make it very easy to write to RTFD and PLIST formats. However, for 2.0 I wanted to make Scrivener’s format less platform-specific. The current format has two main flaws: 1) Its use of .plist and .rtfd files means it’s a format that can only be read on the Mac. Although I personally have no plans to switch to or code for other platforms, this would be a significant hurdle for anyone we wanted to work with to port Scrivener to, say, Windows. 2) The .plist format is not human-readable – at least, not when used with the sort of data that Scrivener has to write out. This makes it difficult for anyone on any platform, including the Mac, to write utilities that might work with the Scrivener format. For these reasons, I am in the process of making the following changes to the .scriv package format: 1) The .scriv package will no longer contain all files in the root folder. Instead, subdirectories will be used to make it easier to navigate. That way, should it be ported to a different platform that doesn’t support packages, it will be easy for the user to find the file required to open the project. And in general it’s just neater, of course. 2) I will no longer use the RTFD format and will switch to using RTF instead. The only reason I didn’t use RTF to begin with was that Apple’s standard RTF reader and writer – the one provided in the Cocoa frameworks – ignores images. That is, it fails to load or save images in the text. Over the years this is something I’ve fixed myself, though, so I use a modified version of the RTF reader/writer to save and load RTF files that retain images with no problems. This not only means that all the text files stored inside a .scriv file are now platform-independent, but also that they are using a file format that has been around for over twenty years. It’s also a format that can be opened in a plain-text reader. 3) Instead of using .plist files, I am creating my own XML file formats where applicable. (There are other changes too – for instance I am now using a checksum file that can tell which files have been changed since the last session; this means that should Scrivener crash, you will no longer be faced with the time-consuming “Synchronising…” panel, as Scrivener will be able to update only the search indexes for the files that have changed rather than going through every single file in the project.) Needless to say, writing my own XML file formats is the most time-intensive part of this process. Fortunately, Cocoa has some excellent and easy-to-use classes for generating and reading XML – the NSXML… classes. For instance, suppose I wanted to create the following XML:
Why I Hate Sky But Quite Like David
Grr, Sky! Grr, Cornwall! (The sky I am referring to is the corporation owned by devil-incarnate Rupert Murdoch*, by the way, and not the most excellent canopy, the brave overhanging firmament, the majestical roof fretted with… Not the actual sky, I mean.)
Stardust and Inciting Incidents
Just to warn those who read this blog purely for Scrivener-related news: this post is only tangential to Scrivener, as it’s about my own fumbling attempts at writing (for which I use Scrivener, obviously), so you can safely tune out if you don’t like reading self-indulgent prattle.
The iPad and Mac Development
So. The iPad. (You may have heard of it. It’s a neat little gadget Apple released last week without much fanfare.) There are commentators out there declaring it the world’s most expensive Etch-a-Sketch (unfair; it has no stylus), and others praising it as being as “magical” and “revolutionary” as Steve Jobs and Jonathan Ive would have us believe. My own opinion on the device is somewhat schizophrenic (in the colloquial sense, obviously). I’m split between my thoughts as a user and my thoughts as a Mac developer.
J.D. Salinger
(Note: Although I've moved most non-Scrivener related posts over to my Machine Dreams blog following a couple of users taking umbrage over my opinions on the objectively terrible Battlestar Galactica finale, I'm making an exception for this one. This is a re-post from the forums, and I feel it's justified purely because J.D. Salinger was one of the authors who made me want to write, and thus had a direct impact on the eventual development of Scrivener. Justification over.)
Dollhouse / Sarah Connor
Okay, so I have now sat through nine episodes of Dollhouse with good grace, trying to trust in the usually brilliant Joss Whedon. After all, he created the generally superb Buffy, the fun Angel (which actually managed to make the hitherto dull character of Angel likeable and played off David Boreanz's comic strengths, now used well in Bones), and the sublime Firefly. (Ah, Firefly and Serenity; I share xkcd's obsession with thee, even though I believe you were derivative of my beloved Farscape.) And it stars Faith my favourite Vampire Slayer and Helo (okay, so I try not to think about BSG too much since the appalling - hock, spit! - finale, but I'll always have a soft spot for Helo and Athena regardless of Ron Moore's tripe ending).
And yet here I am, nine episodes in, and Dollhouse shows no signs of... well, being any good. I hear Alan Tudyk is to appear in the last episodes of the season, so I'm holding on for that, but really, what I don't understand is this: how the hell did this get picked up for a second season by Fox while the excellent Sarah Connor Chronicles was cancelled? The first season of Sarah Connor walked all over Dollhouse with size five Terminator-ballerina boots, and more than earned my patience when I had to put up with a bit of a mediocre middle to the second season (more than pulling it back again for the end of the season). Dollhouse hasn't earned such patience; I'm merely giving it an extended chance because it comes after Firefly.
Does it get any better? It's difficult to care about "actives" who are essentially call girls, especially when all of the cast seem to be turning out to be actives. And yet an emotionless robot played by Summer Glau... Oy, Fox! Bring back TSCC!
And that is my Tuesday lament.
Final Draft 8
For all those scriptwriters out there, and for all those who don’t already know, Final Draft 8 was finally released not long ago. Final Draft is, of course, the industry standard of scriptwriting programs, and version 8 brings with it a raft of new features and an overhaul to the interface that makes it feel much more like a native Mac app than previous versions. I won’t go into the various new features - navigator, improved index card navigation, new interface, and so on - here; instead, I just thought I’d say a few words about Final Draft’s relationship with Scrivener.
Creating TIF files with multiple images
So, in overhauling Scrivener's toolbar graphics and other graphic elements for 2.0, I noticed that a lot of OS X apps handle toolbar images a little more elegantly than Scrivener 1.x. Scrivener's toolbar looks fine when the icon size is set to normal, but if you set the it to use the small image size, the images get rescaled and don't look so hot. A lot of apps - look at Pages for instance - look great at both sizes, because they provide custom images for each rather than just allowing the toolbar to scale the larger images down when the small size option is selected. with Scrivener 1.x, though, I only created images for the larger size.
I was waiting on overhauling images such as these to see if Snow Leopard introduced resolution independence - when that comes just about every image in every OS X app is going to need recreating at a much larger scale by professional artists. But seeing as that doesn't seem to be on the agenda for 10.6 (which makes me sigh with relief as a developer even if the end-user part of me would like to see it), I have started in on overhauling the icon set.
The way OS X toolbars handle selecting the small or large image for a particular toolbar icon is to look in the one image file for both images; that is, it expects both images to be bundled into the same .tif or .icns file (the larger one at 32x32 pixels and the smaller one at 24x24). OS X comes with a tool that will create .icns files easily enough, but being obtuse I decided I wanted to keep the toolbar icons as .tif files (which is how most Apple apps do it). The trouble is, Photoshop doesn't support .tif files containing multiple images. So I Googled around to find a tool which would, but either my search terms were rubbish or the only tools that really do this sort of thing are paid-for, fully-featured apps, and I realised I could write my own tool to do this much more quickly than I could find one from searching through Google results - after all, all it needs to do is take two image files already created in Photoshop, one for the small size and one for the larger size, and bundle them both into the one .tif file.
So, here is my ten-minute app that does exactly this:
http://www.literatureandlatte.com/freestuff/MultiTIFF.zip
It's pretty self-explanatory - you just drag a 32x32 image into the 32x32 image well and a 24x24 image into the 24x24 well, and then hit Save to create a .tif file that combines the two, suitable for use in toolbars.
EDIT: I've updated it so that you can open existing multi-page .tif files and export the small or large icons out as separate files.
Who knows, maybe it will come in useful for somebody else putting their toolbar images together. Probably not; given that a lot of developers do this already, presumably there is already an abundance of tools out there that do this that I just missed, but it was a diverting ten-minute break from the intense coding I'm doing on Scrivener 2.0 at the moment. Which rocks, by the way.
Apple: friendlier
A couple of years ago I got all huffy about how Apple had withheld the Leopard beta released at WWDC '07 from developers who had paid for access to Leopard betas but could not attend WWDC, only finally releasing it to other Apple Developer Connection Select and Premier members about a month later - meaning a lot of developers who had paid up for early access to Leopard were unnecessarily left a month behind WWDC attendees in getting their apps Leopard-ready.
I was not a happy bunny, as many panicked Scrivener users noted in their e-mails to me asking if my beta-rage meant I was likely to abandon Scrivener and the Mac platform (which was never in question). I like to think I have grown and calmed with age in the past couple of years (although my shouting at an ancient Cornish driver the other day may provide evidence to the contrary), so this year, with the coming of WWDC and the announcement that there would be a "near final" build of Snow Leopard available to attendees, I just accepted that I would have to wait another month or so to get access once more. (Despite my vow never to pay for ADC again, it would be a little irresponsible not to do so, as I have to ensure that Scrivener is Snow Leopard-ready for the day of its release... so I reluctantly handed over the cash again this year for the Snow Leopard betas.) Indeed, unlike two years ago, the Apple developer website didn't even claim that ADC Premier and Select members would get the "latest builds", so the signs weren’t exactly auspicious.
So imagine my surprise when I logged into my ADC account the other day to find that the latest, WWDC, build of Snow Leopard was there, ready and waiting for download - and had been there since the 8th June, the very day it was made available to WWDC attendees. I nearly fell out of my chair. Or at least I swivelled in it a bit. I don’t know if there were a glut of other developers such as myself who complained bitterly at Apple two years ago - I certainly didn’t see much complaining about it anywhere else online - or if Apple just decided they should give all paid-up developers equal opportunity to get their apps Snow Leopard-ready, but whatever the reason, given the loudness of my lamentations and weilaways two years ago, I figured I should at least give credit where credit’s due this time around. Okay, it should have been like this in the first place, but at least Apple have got it together now and that makes me (for once) a happy bunny. So, thank you Apple for sorting this out.
So Snow Leopard is now ready on my other system and waiting for me to make sure Scrivener works fine on it (early tests are good), which I will be doing in earnest very shortly, in the midst of the full-on development on 2.0 in which I’m currently immersed. (And to those waiting for news on 2.0... Sorry, but I’m holding back on further announcements or feature peeks until nearer the time. 2.0 is still slated for an end-of-2009 release, and suffice to say that I think a lot of Scrivener users are going to be very happy...)