First things first, if you haven’t already seen this incredible video, may I recommend doing so? You’ll want to load it in HD full-screen mode, let it buffer, then just kick back and relax. (Via the always-awesome Kottke.)

Next up: what’s Omni been doing lately? Let me break it down for you bullet-style:

• Preparing for Snow Leopard! Engineers are madly adding Snow Leopard updates to all our shipping apps, with the goal of having everything ready the day Snow Leopard is. I don’t exactly know how this works, if they just . . . physically stuff an actual leopard in there, or what (aw, who’s a snuggly? You are!), but they seem very busy.

• Starting sneaky peeks of OmniFocus 1.7! LOTS of changes in the works, and you can get the rundown here. Warning: running an alpha version isn’t for everyone, but if you don’t mind, you know, living on the edge a little, the very latest builds are being constantly made available via our sneaky peek page.

• OmniWeb 5.10 sneaky peek updating! Speaking of sneaky peeks (the more I type that term, the goofier it sounds. So SNEAKY, these peeks), the work-in-progress version of OmniWeb, 5.10, has just been updated with the latest WebKit (from Safari 4.0.3) and has several improvements to its built-in software update (including the ability to ignore a particular update).

Questions? Feedback? Cool links to share? Chat us up in the comments.

We’re looking for one or two software development interns for this summer! Check out our open positions for more info on internships or full-time employment.

Out of the box, CALayer supports many animatable properties where a change will automatically create a CAAnimation for that property. But, if you look at the list of properties, it is clear that they are all of the form “stuff that OpenGL does while compositing” and have nothing to do with the inner texture representing your content. You are entirely responsible for telling CoreAnimation when your content has changed.

Recently I wanted to write a layer that had its content animate, but as far as I could determine CALayer doesn’t support this out of the box, though it is really pretty close. If you add an @dynamic property to a CALayer subclass and set it, your layer will be sent -actionForKey: and the returned animation will be passed to your -addAnimation:forKey:.

The problem is that CALayer doesn’t know that this change means it needs to update your contents property too, so the animation will happen but no display change will happen.

The API on CALayer really isn’t sufficient for this task, so it isn’t too surprising it isn’t supported. Since CALayer is a generic KVC container you can squirrel away random keys/value pairs in it and use them for whatever purposes you want. But it has no way of knowing that a particular property change should provoke an update to the content, and updating the content unnecessarily would seriously hurt performance.

I’ve whipped up some sample code with a superclass that shows one approach to handling this.

The sample subclass looks like:

@interface WaveLayer : ContentAnimatingLayer
@property CGFloat phase, frequency, amplitude;
@end

@implementation WaveLayer
@dynamic phase, frequency, amplitude;

+ (NSSet *)keyPathsForValuesAffectingContent;
{
    static NSSet *keys = nil;
    if (!keys)
        keys = [[NSSet alloc] initWithObjects:@"phase",
                   @"frequency", @"amplitude", nil];
    return keys;
}

- (void)drawInContext:(CGContextRef)ctx;
{
    … draw a sine wave …
}
@end

The ContentAnimatingLayer superclass provides some bookkeeping and basic support for updating the content when the content-affecting properties are changed. First, it adds support for -actionFor<Key>; like many other key-based Cocoa protocols (Radar #6372335). On top of this, it adds support for determining which properties are content-affecting with +keyPathsForValuesAffectingContent and -isContentAnimation:. Active content-affecting animations are then tracked and an update timer is fired when there is at least one such animation.

This makes it as easy as I’d hoped it would be to write content animating layers; hopefully you’ll find this useful too! I’ve asked Apple to add this to CALayer in the future. If you’d like it too, you can reference my Radar #6372372.

The sample code is just that, my first cut at a sample. Some possible issues/improvements:

• The -actionFor<Key> support may or may not be fast enough (or really not even necessary).
• Instead of following the class-based KVO customization pattern, CALayers are often customized by the delegate. So, the NSSet-returning method should maybe be an instance method or maybe the -isContentAnimation: should be split up and call a new -layer:isContentAnimatingKey: delegate method.
• It would be nicer to tie into the normal CoreAnimation timing mechanism instead of creating a per-instance NSTimer. Even without that, it might be a bit more efficient to have a single animation timer for all instances of ContentAnimatingLayer. Also, right now the timer doesn’t get registered for the event tracking runloop mode, though it maybe should.
• This approach assumes that a change to a content-affecting key requires the entire content be redrawn. This may or may not be the case for any particular layer. A change to a content-animating property might want to specify an NSRect-based animation that describes the dirty area.

Hello, friends. One of my jobs here at Omni is creating our documentation, including onscreen help in the Apple Help Book format. Over the years I have been building a Python tool that turns specially-formatted OmniOutliner 3 files into proper help books, which can then be dropped into an app. This is pretty useful for “single-sourcing” our help and manuals. If you, too, would like to author your help in OmniOutliner, with automatic formatting, indexing, and navigation, you may want to try it out.

Download Helpify.zip

The included outline acts as documentation and as a starting template. If you have any feedback, or you’d like to help improve my decidedly un-engineer-like code, please let me know at helpify at omni group dot com.

Update: Version 1.1, uploaded October 31, 2008, includes code cleanup, a company web site variable, and the ability to use Helpify as a module for other Python scripts. Many thanks go to Matteo Rattotti of Shiny Frog for his feedback and patience.

Update: Version 1.2, uploaded November 18, 2008, includes a bit more cleanup and much better handling of Unicode throughout the source outline. Thanks to Markus Müller of MindNode for his feedback and patience.

Update: Version 1.5 was uploaded December 21, 2009. See the new blog post for details!

Now that the iPhone NDA is being lifted, we can share a few of the lessons we’ve learned while working with the SDK.

Since OmniFocus makes extensive use of the Omni Frameworks, one of the very first challenges we hit when starting with the SDK was how to structure our source and Xcode projects so that we could re-use some of our battle-tested common code between OmniFocus on the Mac and on the iPhone. Some people like splitting up their source into frameworks and some don’t. That’s all good, but on the iPhone 3rd party developers have no supported way to build their own frameworks.

Many limitations also come with a benefit, and in this case it is smaller application packages. A framework may include classes or categories that are used in several clients, but not all of them. On the iPhone, we want fast downloads from the App Store (possibly over the cell network) and we want fast startup times. Both of these require us to not bloat our executable with code not specifically used by our app.

Given that we can’t use real frameworks bundles, we need to directly include our framework source in our iPhone project. I’ll describe an approach that’s worked well for us and might suit you too.

First, we want to be sure that we don’t pick up any extra headers that we didn’t intend to use. For example, if our OmniFoundation NSSet category imports another extension header, we want to be sure that we consider that change rather than just letting the dependency creep in (if nothing else, we need to build the corresponding .m file). One approach that helps with this is to use <…> instead of “…” imports, so we have:

#import <OmniFoundation/NSSet-OFExtensions.h>

This ensures that Xcode finds our header from a “public” location rather than finding it relative to the file being compiled. Now, if we do have an internal header that shouldn’t be published, we can leave it in an “…” import, but we have to take care to add those .m files to the target.

Second, we want to “publish” the headers for the public headers necessary to build the subset of the framework we’ll be using. We accomplish this with a sequence of Build Phases in our OmniFocus target in Xcode:

The first interesting phase, Create Header Directories, gives us a place in our build area for the headers:

FRAMEWORK_HEADER_BASE="$DERIVED_SOURCES_DIR/FrameworkHeaders"
mkdir -p "$FRAMEWORK_HEADER_BASE"
mkdir -p "$FRAMEWORK_HEADER_BASE"/OmniBase
mkdir -p "$FRAMEWORK_HEADER_BASE"/OmniFoundation
mkdir -p "$FRAMEWORK_HEADER_BASE"/XMLData
mkdir -p "$FRAMEWORK_HEADER_BASE"/OmniFocusModel
mkdir -p "$FRAMEWORK_HEADER_BASE"/OODataTypes
mkdir -p "$FRAMEWORK_HEADER_BASE"/OmniDataObjects

Then, for each “framework” we have a Copy Files build phase that is set to copy to the header directory in question. For example, Copy OmniBase Headers looks like:

Finally, we need to let Xcode find the headers when it builds our target. Opening the Target Info editor for our iPhone app, we can double-click the Header Search Path entry and add the top level FrameworkHeaders directory:

Now we can add just the framework headers and source files we need to our iPhone project. Each “public” header can then be dragged into the appropriate Copy Files build phase for installation during builds.

This isn’t the end of the road, but it is a good start. Other issues involve making sure that your NSError domain strings aren’t dependent on your bundle identifier (since on the Mac they’ll be in different bundles and on the iPhone not), resource location (again, due to the one bundle vs. many difference on the platforms), and generating strings files when your sources are spread around (there are a couple scripts floating around for doing this; once we clean ours up we’ll hopefully publish it).

Dave Dribin posted a nice article on Proper Key-Value Observer Usage that you should definitely check out. Something he mentioned in passing has been bugging me for a while:

You could check the object that’s passed in as well. Unfortunately, this doesn’t help the case where a subclass or superclass has registered a notification on the same key path of the same object.

If, however, you specify something unique for the context, you can use that to correctly identify your notifications.

Proper use of the KVO context will allow a single object to subscribe to a key across a parent and child class in some common cases, but there is a hidden danger. If your two classes both follow the basic pattern of adding an observation when your object is created and removing it when your object is dying, all will be well. But if you have a class that may toggle its observation on and off based on some varying criteria, you can get in trouble.

The core of the problem is an API flaw in KVO; -removeObserver:forKeyPath: doesn’t include a context argument. So, if you have an object that is subscribed to a key twice, with two different context pointers, there is no way to specify which one is to be removed. If your object is dying and cleaning up all observations, this isn’t a big deal. But if your subclass logic wants to temporarily turn off its observation, it has no way to make sure that’s what happens — it might instead end up removing the superclass observation.

Now, granted this is a relatively rare case that can be accounted for in your design. But, this takes an otherwise general and extremely useful API and adds a bit of worry to it every time you use it. You can’t use it locally in a class without worrying about what your super- and subclasses are doing.

I’ve written up Radar 6244260 on this, including a test case. Hopefully they’ll add a proper -removeObserver:forKeyPath:context: and deprecate the current method for some future release.

In April, Matt Neuburg of TidBITS wrote a review of OmniFocus. For those who don’t know Matt, he has some incredible credentials when it comes to reviewing productivity software, particularly in the outlining space: as early as 1993, he wrote some great reviews of Acta and Inspiration, IN CONTROL, and MORE. (And those reviews compare those outliners with the old Apple ][ incarnation of ThinkTank, so clearly it's a space he's been thinking about and working in for a very long time!)

In Matt's review of OmniFocus, he pointed out some of its version 1.0 interface quirks, then said:

With all these gripes, you might think my assessment of OmniFocus would be largely negative. It isn't. I would still insist that OmniFocus is the best expression of GTD on the Mac that I've ever used. Its existence has relieved me of stress and helped me accomplish more in less time.

But those interface quirks did keep him from recommending it without hesitation. We highly value Matt's feedback, and we immediately started reviewing the quirks he noted to see how we could improve OmniFocus in future updates.

Matt's review was great: it not only said what was good about the product now, but also talked about how we could improve it in the future—and at the time, I felt no need to comment on it. But in the last week or two Matt's accompanying screencasts of his frustrations with some of those quirks started getting a lot of additional attention (perhaps because Daring Fireball linked to them), and since lots of people have asked what we think about his feedback I thought I should comment on the quirks he presents in those screencasts. (Please forgive me if this is a bit longer and drier than our usual blogging fare!)

So, here's my summary of Matt's issues presented in his screencasts along with my response to each issue:

• Matt's first concern is with the ghosted field labels which are displayed when the user moves the mouse over different rows. One of our design goals was to avoid relying on fixed column headers, because each row in OmniFocus has its own set of fields. (For example, rows representing inbox items have both a project field and a context field, but rows representing projects have neither.) For the first several months of the beta we didn't even have the option of headers, but we ultimately decided to introduce them as a way to directly resize fields. But they're still hidden by default because they change based on the current selection (since each row has its own set of fields).

  Since we didn't have column headers, we started thinking about how to help the user understood which field did what (which is the start date and which is the due date?), and for inspiration we looked to iTunes. In iTunes, the selected row always has a ghost rating field visible so that you'll know where to click to assign a rating.

  We've applied a similar model to our fields in OmniFocus, adapted to highlight whichever row is under the mouse because you don't have to select a row before you can edit it. So, when you move the mouse over a row, a ghosted light gray flag icon appears to indicate where you can click to flag an item—and if you show the optional Due date field, a ghosted "Due" label appears in it when it's empty so you'll know what that field is for. We've tried to tune this highlighting to be pretty subtle and yet still be visible when you are looking where the mouse is currently pointed, but we agree that it's not perfect and we're continuing to improve it.

  Matt's screencasts expressed some concern that the user might be confused over whether the ghost flag was real or not, but they didn't show what happens when either of those fields actually has some content: the flagged column has a bright orange flag, and the due date column shows an actual date. (And, of course, the entire row turns red when something is actually due!)

  We now have tens of thousands of customers using OmniFocus, and most of the feedback we've received seems to indicate that this arrangement is working quite well for most of them. But we're still working to improve it, and we're always willing to listen to alternative suggestions!

• The next quirk was that we were using a non-standard combo box field for our projects and contexts. It's certainly true that it's not a standard field: we want the user to be able to type a few letters to quickly select a project or context: for example, unlike a standard combo box I can type "dofip" or "omnif" to select my "Develop OmniFocus for iPhone" project. This is an important feature when you're trying to get something out of your head quickly!

  Unfortunately, as Matt points out, those fields do currently have a bug: there's no good way to cancel a selection if you change your mind about your entry. You can back out and empty the field using the Delete key, but as he suggests you should also be able to press Escape or click away from the field to cancel input—and unfortunately those actions currently accept your input. (You can undo your change, of course, but undo shouldn't be a replacement for cancel.)

  But other than that bug (which we're planning to fix), we've been very careful to emulate the standard Cocoa combo box behavior with respect to mouse activation, clicks, and drags. The other behaviors he described are the way standard Cocoa controls behave: when you drag down on a combo box, the highlight follows the mouse and releasing the mouse pointer selects whichever item is currently highlighted. But if you instead click on the combo box to open the list (which is what he did), mouse movements are ignored until you click on an item to choose it.

• Similarly, Matt is concerned about issues with the calendar widget, stating that these problems are all caused because we've "designed these bad widgets" rather than just using the stock Cocoa widgets that "just work." We totally agree with him about the misbehaviors of the widget itself, but the irony here is that we're actually using the stock NSCalendarView control. We didn't write our own; we have no desire to write our own. (But we do want it to work right, so perhaps we'll have to write one to fix some of those issues!)

  (As an aside, he could have cancelled his selection by just clicking back on the same calendar icon he used to open that calendar, rather than moving the mouse first. But we still agree that clicking somewhere else should also have cancelled the calendar selection.)

• On to the second screencast! Matt's first issue in this screencast is that there's no visible focus when nothing is selected. (Matt didn't demonstrate this, but this is only an issue when there's no selection: if there's one or more items selected in either the sidebar or the main outline, you can tell which has focus because its selection appears in your highlight color rather than in gray.) Well, we agree, that's a problem, and unfortunately this is a common issue in Mac applications when there's no selection. (Mail has the same issue with its sidebar, for example.) I haven't yet seen a good solution for this, but if you have I'd love a pointer!

• Matt's next issue was with our column resizing behavior: "Have you ever seen a Mac program where when you widen and narrow a column the entire window widens and narrows like that?" Well, actually, I'm guessing you have, since we borrowed that behavior from the Finder! If you open a Finder window and switch to Column view, then drag the last column wider than the window and then narrower again, you'll find that it makes the whole window wider and then narrower.

  Now, just because there's precedent in the Finder for a window resizing along with a column doesn't mean I'm saying that this is the most desirable behavior. But what is? What exactly do you expect to happen when you make the title field wider or narrower? Should we just pick another field at random to make wider or narrower to compensate for that change? Or should the column widths no longer match up with the width of the window, leading to lost fields off the edge or to dead space on the inside of the edge? Most approaches we've seen lead to lots of fiddling to try get all the columns to add up to just the right width to fit within the window.

  [EDIT: Expanding on this a bit to explain the problems with other approaches in more detail.]

  We considered Finder’s list view behavior (which we also use in OmniOutliner) of introducing a scrollbar, but that’s not perfect either: in OmniFocus, everybody wants to see all the fields all the time, not some subset. (This is especially true because we have different fields on different rows.) If we hide some of those fields behind a scrollable area, the very next thing the user will want to do is to try to make the window fit the new width of the columns, which (as I mentioned above) can take a lot of fiddling. We were trying to save people this extra fiddly step by making the window fit at all times.

  We also considered Mail’s solution: Mail solves the “exact width” fiddling problem by taking the size away from another field. But with that approach, making the project field wider would make the context field narrower, which isn’t usually what people are trying to accomplish. (If they finally got their Context field just the right size to fit all their contexts, they don’t really want it getting smaller just because they now need to make their Project field a little wider!)

  [END EDIT]

  Maybe a preference is in order here? Again, suggestions for how to solve this are welcome! Just remember, one important design principle is that the edge that you’re resizing should stay under the mouse, not wander off somewhere else.

• Next, Matt demonstrates a bug where a row beeps if you double-click on its handle—which, yes, is a bug. We’ll fix it! (Double-clicking on a project opens it in a new window, but you can’t open an individual action in its own window so that currently beeps. What it should do is fall back on the single-click behavior, not beep!)

• Matt points out that it’s very unclear exactly where to click to expand or collapse a row rather than selecting or dragging it. We agree that it’s hard to tell exactly which pixels will do what, and I think the easy solution here would be for us to give better mouse pointer feedback (like we do when you mouse over the resize bars in the column headers).

  (As an aside, you don’t have to click precisely on the little black dot on the left to drag a row around: you can actually drag a row vertically from anywhere on the row: that handle, or the checkbox, or even the text itself.)

• Finally, Matt asks what you can do to select nothing. Well, good question! The way to select nothing in OmniFocus is exactly the same way as you select nothing in the Finder or iTunes or Mail: you either click on some empty space that isn’t selectable (which isn’t always easy to find, unfortunately), or you command-click on your current selection to toggle it. (I wish this were more obvious, but this is the standard Mac behavior.) [EDIT: Chris Campbell's comment below made me realize that OmniFocus does have a Deselect All command in the Edit menu, with a keyboard shortcut of Shift-Command-A.]

Let me conclude by saying once again that I have a lot of respect for Matt’s feedback: he’s been writing about this application category for a very long time, and he’s pointed out some important issues which we hope to address as soon as possible. Our goal is to fully earn his final words from the review:

I’ve raved in the past about Omni’s interfaces; OmniGraffle is brilliant for drawing diagrams, and OmniPlan is an astounding accomplishment, a triumph of interface ingenuity and the first project management application I’ve come even close to comprehending. I’ve little doubt, and much hope, that the same standards of excellence can be applied to OmniFocus; when OmniFocus has the fluid usability of Omni’s other applications, I’ll be eager to recommend it.

Thanks, Matt: we’re working on it! (And thank you, reader, for taking the time to read all this; if you have any thoughts of your own, we always welcome your feedback! The best way to make sure the right people see your suggestions is to send them in email to omnifocus@omnigroup.com.)

Bill Bumgarner has posted a few great articles on Objective-C method tracing using dtrace:

• Objective-C: Printing Class Name from Dtrace
• Objective-C: Using Instruments to trace messages-to-nil

I have a internal tool at Omni that traces message sends by creating new method IMPs to wrap the existing ones in a trampoline, but it requires some nasty assembly stubs and has only ever worked on Sparc, ppc and x86. It is very fast, but hard to maintain and a little fragile (and doesn’t handle the nil case). A 90% solution can be obtained by extending Bill and other’s work.

Ken sent out a partial tracing solution this morning and finally got me off my keister. Here is a D script that catches ObjC message sends and dumps a OPML fragment (which can then be wrapped in the XML goo via an external shell script or whatever) and then viewed in OmniOutliner.

Download: objc-trace-opml.d.gz

This isn’t a perfect solution by any means. Some of the issues:

• This has to be run in an essentially single-threaded program; any background threads must be totally quiescent or you’ll get a mixture of output from multiple threads. It wouldn’t be too hard to modify this script to include a thread identifier or maybe just filter out anything from background threads.
• This won’t handle exceptions correctly; they’ll break the OPML nesting. Our internal tool handles this, but since NSError arrived on the scene, this is less of an issue. Just avoid tracing code with exceptions.
• dtrace is slow when matching all entries in the objc provider. The startup time on this script is something like 40 seconds on my machine. Annoying, but not the worst thing ever, since you are likely to trace once and then spend a fair bit of time examining the results.
• This particular variant of the script will only work on x86 due to the hacky way I check for objc_msgSend_stret.
• dtrace can overflow its buffer and drop events when there are massive numbers of probe hits. Run with a big buffer and trace only the exact set of methods you need. Ideally you’ll be running in the debugger, stop right before where you need to trace turn on tracing and then ‘next’ over one or two lines of code. The ‘-b’ flag to dtrace can also be used to increase the buffer size.

Some of the nice bits:

• No assembly-fu required.
• Call hierarchy is preserved and can be expanded/hoisted and such in OmniOutliner.
• Both the class of the receiver and the class of the method implementation are included. So, you can see that +[MySuperclass initialize] is getting called on MySubcass, for example.
• All method invocations are shown, so you can see nested calls to super instead of just the initial call to objc_msgSend. I’ve not tested whether cached method IMPs get traced too; normal code should get traced as expected. Swizzling, IMP caching or dynamically registering methods will possibly not work as well.
• The pointer value of the receiver is emitted. By the time you examine the output, it might be dead and gone, but a surprising amount of the time it isn’t. For example, things like static NSStrings, NSScriptClassDescriptions, interface widgets and other cache-once data may still be around for submitting to ‘po’ in the debugger.

We finally have space to grow, now that we’ve moved to our new office! There is no specific position to fill, but rather we are content to wait until we find someone who will fit well into our team. Take a look at what working at Omni is like and the general description of who we are seeking to see if this is the right place for you!

I failed to find any way to do this via the internets; but gdb, and some kindly soul inside Apple, met the challenge:

(gdb) b -[NSUserDefaults objectForKey:]
Breakpoint 17 at 0x94172fa4
(gdb) c
Continuing.
... invoke service, hit breakpoint ...
(gdb) po *(id *)($fp + 16)
NSShouldActivateForServiceRequest
(gdb)

Is there a better way to do this? Google and mdls don’t seem to know about NSShouldActivateForServiceRequest at all, so I’m guessing not.