FreeBSD core team member Scott Long posted the latest project status report, covering the last 6 months of development. Scott begins:

"The FreeBSD status report is back again after another small break. The second half of 2004 was incredibly busy; FreeBSD 5.3 was released [story], the 6-CURRENT development branch started [story], and EuroBSDCon 2004 was a huge success, just to name a few events. This report is packed with an impressive 44 submissions, the most of any report ever!"

Read on for the latest status report, conveniently divided into several sections discussing various FreeBSD pojects, documentation changes, kernel changes, new and updated architectures, the status of various ports, vendor or 3rd party software news, and other miscellaneous news.

Jeff Roberson, the primary developer of FreeBSD's revived ULE scheduler [story], has committed a new a python/tkinter tool, schedgraph, to the freebsd -current tree. Schedgraph will assist with scheduler testing and refinement as well as help developers study application load and corresponding system behavior. In the initial commit message, Jeff gives a simple description:

"Schedgraph takes input from files produces by ktrdump -ct when KTR_SCHED is compiled into the kernel. The output represents the states of each thread with colored line segments as well as colored points for non-state scheduler events. Each line segment and point is clickable to obtain extra detail."

Jeff includes a screenshot and sample data. Robert Watson follows up with a link to pointers on getting KTR working. Jeff has also been very busy committing more fixes to the ULE scheduler, a couple of which solve long standing bugs. Read on for details.

Early last week, Scott Long publicly requested heavy testing of the latest revisions to -current code, especially the new vnode layer [story], net80211 [story], and wireless lan drivers. Now, with the revival of the ULE scheduler [story], stress testing is likely even more urgent. Scott explains:

"There has been an incredible amount of work going into 6-CURRENT in the last 5 weeks. With the holidays quickly approaching, now seems like a good time to step back and start really testing the system to shake out the bugs and catch the developers before they disappear for the holidays."

Read Scott's full message below.

Since the decision to demote ULE [story] in favor of the 4BSD scheduler as the default for FreeBSD's 5.3-Release, many improvements to both schedulers have been committed. At the time it was marked broken, ULE was especially needy in light of the status of its maintainership, performance issues, and its unreliable nature in conjunction with threading and kernel preemption. Having resolved these problems, Jeff Roberson announces to -current that the ULE code is now in working order:

"ULE works again with preemption and kse and so on. As far as I know, the only two problems with ULE currently are these:[...]"

Note that Jeff follows up his initial post with a correction (there are three current problems), and later, that he's fixed the second listed issue. Also discussed are plans to MFC this work to the RELENG_5 stable branch and future direction of ULE and 4BSD scheduler development.

Andre Oppermann has posted a patch which completely revises FreeBSD's TCP reassembly code, increasing efficiency and scalability of fragmented packet reassembly with impressive initial test results. Oppermann writes:

"I've totally rewritten the TCP reassembly function to be a lot more
efficient. In tests with normal bw*delay products and packet loss
plus severe reordering I've measured an improvment of at least 30% in
performance. For high and very high bw*delay product links the
performance improvement is most likely much higher."

Those with spare cycles may want to give it a try. Read on for the entirety of Oppermann's educational post and a link to the patch.

Sam Leffler has recently committed a major 80211 update to FreeBSD's -current tree, providing full support for Wi-Fi Protected Access (WPA), 80211i, and 802.1x among other new features such as the QoS WME/WMM protocols. Sam suggests the new code is stable and tested, but advises:"[...] I expect that wi cards operating in hostap with wep will need some fixup. If you encounter problems please post
here as I've been promised other folks will assist in handling issues." Sam will also be committing revisions to ifconfig to support the new 80211 code, as well as new dhclient code that understands 80211 layer events.

Apropos to these changes, Leffler has also updated the ath driver to take advantage of the new 80211 layer functionality, among other things. See the links section below.

Scott Long recently submitted a list of "nice-to-have" projects for the next 12-months and future releases in general. Many comments and addendums are suggested by various developers and committers. Topics including the adoption of Dragonfly BSD's extensible bsdinstaller (among others) as a worthy sysinstall replacement, ongoing development of ReiserFS4 support for FreeBSD, interesting discussion concerning clustered filesystem support, comments on SANs and much more.

"Most of these tasks are
not trivial, but I hope that talking about them will encourage some
interest. [...]While this is just my personal list, I'd welcome
other additions to it (in the sense of significant projects, not just
individual PRs or bug fixes that one might be interested in)."

Recent development patches committed to FreeBSD's HEAD (-current) and RELENG_5 branches as well as recent patches submitted to the -current, -arch, -threads, -hackers and -net lists need testing in a variety of system configurations. As many know, the quality of future FreeBSD releases is in no small part dependent on user feedback, and FreeBSD developers need feedback on their patches to ensure feature stability and reliability in core subsystems. At present, many of the most important changes in -current surround removing GIANT remnants from various subsystems to make them multi-processor safe, continued integration and improvement of fine-grained locking and ACPI/APIC code. Those users running -current may be interested in stress testing some recent code and reporting findings back to the respective mailing lists.

Jeff Roberson summarizes in his recent VFS patch:

"This patch removes Giant from the read(), write(), and fstat() syscalls,as well as page faults, and bufdone (io interrupts) when using FFS. It adds a considerable amount of locking to FFS and softupdates. You may also use non ffs filesystems concurrently, but they will be protected by Giant. If you are using quotas you should not yet run this patch. I have done some buildworlds, but any heavy filesystem activity would be appreciated."

Read on for a quick round-up of recent patches, the authors original posts and links to their respective threads. This is not a complete list, so please comment if we've missed any!

While FreeBSD -current is still moving toward more stable footing, many users have posted issues with panics and deadlocks in recent kernel builds. Bjoern A. Zeeb has kindly compiled a running list of lock order reversals, links to relevant threads, PRs, and existing patches. Lock order reversal messages are the result of FreeBSD's lock validation facility, witness(4), notifying the system of potential deadlocks as a means for developers to isolate bugs. Robert Watson explains in a Dec. 2003 thread:

"[...]Among other things, Witness performs run-time lock order
verification using a combination of hard coded lock orders, and run-time
detected lock orders, and generates console warnings when lock orders are
violated. The intent of this is to detect the potential for deadlocks due
to lock order violations; it's worth observing that Witness is actually
slightly conservative, and so it's possible to get false positives...."

Read on for relevant links.

FreeBSD core team member Scott Long posted the latest bi-monthly status report, covering FreeBSD development for May and June of 2004. Scott begins:

"May and June were yet again busy months; the Netperf project passed major milestones and can now be run with the debug.mpsafenet tunable turned on from sources in CVS. The ARM, MIPS, and PPC ports saw quite a bit of progress, as did several other SMPng and Netgraph projects. FreeBSD 5.3 is just around the corner, so don't hesitate to grab a snapshot and test the progress!"

The code freeze for FreeBSD 5.3 [story] is scheduled to begin on August 15. The current todo list can be found on the FreeBSD project website. Read on for the latest status report, covering a wide range of FreeBSD projects.

Marcel Moolenaar has been very busy with GDB code as of late, having imported gdb version 6.1.1 in late June and now supplying a patch to freebsd-arch@ that adds kernel debugging and helpful features to FreeBSD's gdb and ddb code, including thread awareness. Other interesting additions include optimizations for the 64-bit platforms, compression for remote gdb, and improved symbol handling. Marcel is looking to commit the patch in a week, barring any major issues, but the code requires testing on most supported platforms:

"The patch applies to alpha, amd64, i386, ia64 and sparc64. amd64 is known to compile but I can't test this stuff yet due to lack of hardware."

Completion of this work will satisfy a required feature on the 5.3 Open Issues list. Read on for the diff and Marcel's full announcement.

Native Preemption in the ULE kernel was introduced into FreeBSD -current on July 2nd by Robert Watson. In line with the purpose of the -current branch, this code still requires some testing and has shown some early issues which have been solved in part. This change essentially allows the kernel to avoid some context switching overhead.

The latest release of the FreeBSD -STABLE branch, 4.10-RELEASE,
is now available. Historically updates to the -STABLE branch have been
limited to bug and security fixes, however, this release marks the first
release of the "Errata Branch" which will contain some additional,
well-tested updates. The announcement indicates that they expect do one
more release of the Errata Branch, 4.11-RELEASE, prior to the much
anticipated 5.3-RELEASE.

For a complete list of changes see the
release notes, which include a "what's new" section.
For a complete, up-to-date list of known problems and news see the
errata list.

Robert Watson maintains a repository of experimental network performance patches intended to improve locking in the FreeBSD -current socket layer. Derived from the work of Scott Long and Sam Leffler among others, these patches require stability testing and performance analysis in uniprocessor and multiprocessor configurations, specifically using network drivers that are MP-safe. Currently, those drivers are: ath(4), bge(4), em(4), ep(4), fxp(4), sn(4), wi(4), sis(4), and rl(4).

On May 18, in an announcement to freebsd-performance@, Pawel Dawidek posted the results of recent netperf patch testing using the em and bge drivers on UP and MP systems with a variety of kernel debug options. Performance of the ULE and 4BSD schedulers were also tested in these configurations.

Read on for more.

FreeBSD Core's Scott Long has posted the latest FreeBSD Status Report to the freebsd-current@ mailing list. FreeBSD Status Reports are semi-regular updates on work-in-progress, new developments and notes from code maintainers. Progress between March and April has been substantial - the 4.10-Release is close to wrap, 5.3 is on target for a summer release as well as positive news for a -stable branch from 5.3. Among many other interesting highlights:

• New OS Internals Book: Kirk McKusick will be releasing The Design and Implementation of the FreeBSD Operating System, a followup to The Design and Implementation of 4.4BSD. The new text will cover the architecture and subsystem internals of FreeBSD-5 (incl. changes up to 5.3). It's in final production from Addison-Wesley and will be available in August of this year.

• Network Stack Locking: Network performance patches are reaching stability although locking issues in IPv6, Kqueue [story], and ifnet areas still need work. Testing and performance measurement is needed. Those interested in participating should see Robert Watson's Netperf Repository. Be sure read the associated notes further down on the page (after the file listing).

• Threading Support: Progress of Thread Local Storage is looking promising albeit a lot this work is taking place behind the scenes. Doug Rabson has a written a set of patches implementing TLS but there have been no specifics on when those will be made available to the public.

• SMPng: Giant is still being pushed down in various areas including the VM and and several system calls incl. fork(2),exit(3), and wait(2). Rudimentary interrupt storm protection has been introduced as well as optimizations to reduce context switching under heavy load.

Read on for a wide range of updates including improved ACPI support, PCI powerstate and resource allocation work, libarchive, porting of OpenBSD's pf, MAC improvements, the new ARM port and much, much more!