The default diff format used by darcs makes sense for text files, but usually not for binary files. Therefore, patches to binary files simply consist of the content before the change, and the content after the change.
Which files are treated as binary is specified in the file _darcs/prefs/binaries. Each line, except those starting with #, is treated as a regexp matching binary files.
The original darcs client supports choosing another file as the list of binary regexps, and keeping this file under version control in the tree. cl-darcs doesn't yet support that.