Content from 2018-12
"Yule" progress report
posted on 2018-12-31 23:59
Dear Community,
Winter solstice is a special time of year when we gather together with people dear to us. In pagan tradition this event is called "Yule". I thought it is a good time to write a progress report and a summary of changes made since the last release. I apologise for infrequent updates. On the other hand we are busy with improving McCLIM and many important (and exciting!) improvements have been made in the meantime. I'd love to declare it a new release with a code name "Yule" but we still have some regressions to fix and pending improvements to apply. We hope though that the release 0.9.8 will happen soon.
We are very excited that we have managed to resurrect interest in McCLIM among Common Lisp developers and it is thanks to the help of you all - every contributor to the project. Some of you provide important improvement suggestions, issue reports and regression tests on our tracker. Others develop applications with McCLIM and that helps us to identify parts which need improving. By creating pull requests you go out of your comfort zone to help improve the project and by doing a peer review you prevent serious regressions and make code better than it would be without that. Perpetual testing and frequent discussions on #clim @ irc.libera.chat help to keep the project in shape. Financial supporters allow us to create bounties and attract by that new contributors.
Finances and bounties
Speaking of finances: our fundraiser receives a steady stream of funds of approximately $300/month. We are grateful for that. Right now all money is destined for bounties. A few times bounty was not claimed by bounty hunters who solved the issue – in that case I've collected them and re-added to funds after talking with said people. Currently our funds available for future activities are $3,785 and active bounties on issues waiting to be solved are $2,850 (split between 7 issues). We've already paid $2,450 total for solved issues.
Active bounties:
- [$600] drawing-tests: improve and refactor (new!).
- [$600] streams: make SEOS access thread-safe (new!).
- [$500] Windows Backend.
- [$450] clx: input: english layout.
- [$300] listener: repl blocks when long process runs.
- [$150] UPDATING-OUTPUT not usable with custom gadgets.
- [$150] When flowing text in a FORMATTING-TABLE, the pane size is used instead of the column size.
Claimed bounties (since last time):
- [$100] No applicable method for REGION-CONTAINS-POSITION-P -- fixed by Cyrus Harmon and re-added to the pool.
- [$200] Text rotation is not supported -- fixed by Daniel Kochmański.
- [$400] Fix Beagle backend -- cancelled and re-added to the pool.
- [$100] with-room-for-graphics does not obey height for graphics not starting at 0,0 -- fixed by Nisar Ahmad.
- [$100] Enter doesn't cause input acceptance in the Listener if you hit Alt first -- fixed by Charles Zhang.
- [$100] Listener commands with "list" arguments, such as Run, cannot be executed from command history -- fixed by Nisar Ahmad.
- [$200] add PDF file generation (PDF backend) -- fixed by Cyrus Harmon; This bounty will be re-added to the pool when the other backer Ingo Marks accepts the solution.
Improvements
I'm sure you've been waiting for this part the most. Current mid-release improvements and regressions are vast. I'll list only changes which I find the most highlight-able but there are more and most of them are very useful! The whole list of commits and contributors may be found in the git log. There were also many changes not listed here related to the CLX library.
- Listener UX improvements by Nisar Ahmad.
- Mirrored sheet implementation refactor by Daniel Kochmański.
- New demo applications and improvements to existing ones,
- Font rendering refactor and new features:
This part is a joint effort of many people. In effect we have now two quite performant and good looking font rendered. Elias Mårtenson resurrected FFI Freetype alternative text renderer which uses Harfbuzz and fontconfig found in the foreign world. Daniel Kochmański inspired by Freetype features implemented kerning, tracking, multi-line rendering and arbitrary text transformations for the native TTF renderer. That resulted in a major refactor of font rendering abstraction. Missing features in the TTF renderer are font shaping and bidirectional text.
- Experiments with xrender scrolling and transformations by Elias Mårtenson,
- Image and pattern rendering refactor and improvements by Daniel Kochmański.
Both experiments with xrender and pattern rendering were direct inspiration for work-in-progress migration to use xrender as default rendering mechanism.
Patterns have now much better support coverage than they used to have. We may treat pattern as any other design. Moreover it is possible to transform patterns in arbitrary ways (and use other patterns as inks inside parent ones). This has been done at expense of a performance regression which we plan to address before the release.
- CLX-fb refactor by Daniel Kochmański:
Most of the work was related to simplifying macrology and class hierarchy. This caused small performance regression in this backend (however it may be fixed with the current abstraction present there).
- Performance and clean code fixes by Jan Moringen:
Jan wrote a very useful tool called clim-flamegraph (it works right now only on SBCL). It helped us to recognize many performance bottlenecks which would be hard to spot otherwise. His contributions to the code base were small (LOC-wise) and hard to pin-point to a specific feature but very important from the maintanance, correctness and performance point of view.
- Text-size example for responsiveness and UX by Jan Moringen,
- Various gadget improvements by Jan Moringen,
- Box adjuster gadget rewrite by Jan Moringen:
clim-extensions:box-adjuster-gadget deserves a separate mention due to its usefulness and relatively small mind share. It allows resizing adjacent panes by dragging a boundary between them.
- New example for output recording with custom record types by Robert Strandh,
- PostScript and PDF renderer improvements by Cyrus Harmon,
- Scrigraph and other examples improvements by Cyrus Harmon,
- Multiple regression tests added to drawing-tests by Cyrus Harmon,
- Ellipse drawing testing and fixes by Cyrus Harmon,
- Better contrasting inks support by Jan Moringen,
- Output recording and graphics-state cleanup by Daniel Kochmański,
- WITH-OUTPUT-TO-RASTER-IMAGE-FILE macro fixed by Jan Moringen,
- Regions may be printed readably (with #. hack) by Cyrus Harmon,
- event-queue processing rewrite by Nisar Ahmad and Daniel Kochmański:
This solves a long standing regression – McCLIM didn't run correctly on implementations without support for threading. This rewrite cleaned up a few input processing abstractions and provided thread-safe code. SCHEDULE-EVENT (which was bitrotten) works as expected now.
- Extensive testing and peer reviews by Nisar Ahmad:
This role is easy to omit when one looks at commits but it is hard to overemphasize it – that's how important testing is. Code would be much worse if Nisar didn't put as much effort on it as he did.
Plans
Before the next release we want to refactor input processing in McCLIM and make all stream operations thread-safe. Refactoring input processing loop will allow better support for native McCLIM gadgets and streams (right now they do not work well together) and make the model much more intuitive for new developers. We hope to get rid of various kludges thanks to that as well. Thread-safe stream operations on the other hand are important if we want to access CLIM application from REPL in other process than the application frame (right now drawing from another process may for instance cause output recording corruption). This is important for interactive development from Emacs. When both tasks are finished we are going to release the 0.9.8 version.
After that our efforts will focus on improving X11 backend support. Most notably we want to increase use of the xrender extension of clx and address a long standing issue with non-english layouts. When both tasks are accomplished (some other features may land in but these two will be my main focus) we will release 0.9.9 version.
That will mark a special time in McCLIM development. Next release will be 1.0.0 what is a significant number. The idea is to devote this time explicitly for testing, cleanup and documentation with a feature freeze (i.e no new functionality will be added). What comes after that nobody knows. Animations? New backends? Interactive documentation? If you have some specific vision in which direction McCLIM should move all you have to do is to take action and implement the necessary changes :-).
Merry Yule and Happy New Year 2019
This year was very fruitful for McCLIM development. We'd like to thank all contributors once more and we wish you all (and ourselves) that the next year will be at least as good as this one, a lot of joy, creativeness and Happy Hacking!
Sincerely yours,
McCLIM Development Team
