KernelTrap: New Home At The Open Source Lab

Submitted by Jeremy
on May 6, 2005 - 5:32am

KernelTrap has found a new home with Oregon State University's Open Source Lab. Associate Director of the OSL, Scott Kveton explained, "the OSL staff were all avid readers of KernelTrap.org and we wanted to do something for this great site. In addition, we really wanted to help a kernel-related site that focuses on the community and community-related issues." Working with Scott and the others at the OSL has already proven to be a rewarding experience.

For the past two years, all colocation costs for KernelTrap have been generously donated by Danube Technologies, Inc.. Danube president, Laszlo Szalvay said, "it was a pleasure hosting KernelTrap for the past few years; we really enjoyed having them on board and are glad to help the open source community." Danube began providing flawless hosting for KernelTrap in August of 2003, however since that time they have refocused their energies, moving away from providing hosting. Laszlo explained, "Danube Technologies, Inc. provides software development services including staff augmentation, project outsourcing, and process mentoring and training." During the search for and later the implementation of our new hosting, Danube has remained excellent to work with.


The Open Source Lab
The Open Source Lab was officially formed in January of 2004, though the team was informally doing what it does for many years before that. When I asked Scott Kveton what the Open Source Lab is, he replied, "in a nutshell, it's about community." He went on to explain, "innovation is happening at a tremendous pace in the open source community. The goal of the Open Source Lab is to bring FOSS communities together and so by doing promote cross-pollination of ideas and people to help create an atmosphere of innovation around open source." The OSL home page offers:

"The Open Source Lab at Oregon State University exists to help accelerate the adoption of open source software across the globe and aid the community that develops and uses it. The OSL's talented team of students and full-time staff do this by focusing on software development as well as hosting the world's largest open source projects."

As to why the open source lab provides hosting, "so often we depend on projects here at OSU that are hosted under some persons' desk behind a DSL line," Scott said. "We wanted to give back to the open source community that has given us so much, and hosting was one of the best ways to do that."

My original communication with the Open Source Lab was with OSL Associate Director Scott Kveton. Scott has been with Oregon State University as a student and a full-time employee for over 10 years. He was away from OSU for about two and a half years while he worked for various dot coms, but ultimately found he preferred the University life, "plus my wife didn't like my dot com work schedule," Scott added with a laugh. "I love my job here; I'm the luckiest guy in the world. I get to work with the open source community and try to help accelerate the adoption of open source as well as decrease the transaction time of developer innovation to end-user experience which is just exhilarating." With no shortage of good things to say, Scott added, "the best part about my job is the team I work with; we have an unbelievable group of talented people here that make this such a fun place to work."

On that team is Corey Shields, the Infrastructure Manager. I've been working with Corey the past couple of weeks to get a temporary server configured for use by KernelTrap. Corey has been working with Linux for ten years, saying, "since then I've loved the community aspect as much as the end-user experience of Open Source Software." Prior to working for the OSL, he worked at Indiana University maintaining their ftp mirror, "they have now grown to serving up quite a bit of bandwidth for OSS, and hosting quite a few servers for Gentoo Linux." He went on to add, "I came to the OSL this year because of the opportunities to work with even more OSS communities, and do what I can to help out. It's a great job!"

OSL Funding
Scott Kveton first approached me nearly two years ago, the last time KernelTrap was looking for a new host. At that time, KernelTrap had already begun our excellent relationship with Danube Technologies, so I declined the offer to move KernelTrap's server into OSL's lab. Scott kindly followed up a year later, making it clear that should anything come up, the offer was still there. Thus, when Danube indicated that they were planning to stop providing hosting, Scott was the first person I talked to.

I wanted to understand how OSL gets its funding, hoping to put the KernelTrap server somewhere that it will be able to stay for a long time to come. When I asked Scott about OSL's funding, he laughed, "I swear I should have a t-shirt that answers this ... so many people ask it ... :-)" He went on to explain, "OSU's information services group had a bit of a financial disaster 8 years ago; our former CIO went a little crazy writing checks, etc. After the dust settled, we were left with a huge debt, and increasing user base and the need to provide a lot more with a lot less. We had no choice but to turn to open source."

Scott continued, "well, 7 years later we paid down the debt and had turned our infrastructure into an open source powerhouse; students at OSU get loads of disk space, PHP for their websites, blogs and they even get their own MySQL database. After paying down the debt, we had 'strategic' dollars to spend for moving information services forward. Our CIO (Curt Pederson) chose to fund a fiber build-out to I-5 and the Open Source Lab. The strategic funds are seed money to get the OSL off the ground and we're well on our way to cost recovery. All of our development projects are cost recovery and our hosting will be completely cost recovery by the end of 2005."

Project's Hosted by OSL
Scott explained that most projects that are currently hosted by the OSL arrived there by word-of-mouth, "such-and-such project had a machine that died and needs help or so-and-so is reaching the limits of their existing infrastructure and needs help." The OSL staff then votes to decide if the new project meets the following criteria:

  • "Are they community focused?
  • "Do they have committed, energetic and most importantly realistic leadership?
  • "Will this project's community interact well with our current hosted partners?
  • "Will the OSL's involvement in this project help its community grow and will we be able to increase the speed at which innovation is adopted by this community?"

Among the projects that are hosted by the OSL is much of the Gentoo Linux infrastructure. "we originally did this because they needed help, but more because they had an amazingly vibrant community. We wanted to help them in any way that we could." Through their relationship with the Gentoo project, OSL became aware of Freenode's capacity problems, and offered them hosting as well.

Another large project, much of the Mozilla Foundation's external infrastructure is also hosted by the OSL. Scott used this example to explain how many of the hosted projects work together, "during the 1.0 release of Firefox, the Gentoo community helped by tweaking some machines we had here at the OSL to get the MoFo website back on-line." Not only did the Gentoo Project help to restore the Mozilla Foundation server, but some of the resulting performance patches were actually merged into the amd64 tree. Scott exlaimed, "_that_ is frickin' cool."

The OSL hosts and works with an impressive list of projects. The list includes the Mozilla Foundation, Debian GNU/Linux, Gentoo Linux, Freenode.net, Arklinux, the PowerPC Kernel Archives, and SPI.

OSL Infrastructure
The Open Source Lab has plenty of bandwidth. Scott summarized, "we currently have 2 OC48's out of Corvallis; one for commodity and one for Internet2. Our commodity providers are Level3 and WilTel in Portland and Sprint in Eugene." Scott offered the details, "we have 96 pairs of fiber to I-5 that allow us to tap into dark fiber that gives us connectivity to Portland and Eugene (the major POPs in Oregon). NERO provides our connectivity to commodity and Internet2 so we just get an up-link from them and they leverage our fiber to I-5. It's a win-win." Corey Shields noted, "our location is key," pointing out that they're only 15 miles away from the dark fiber along Interstate-5, "fiber was laid a couple of years ago to tap into this, and we are now set for future growth."

The OSL lab itself is currently housed in the campus Information Services facility, as it has been for many years. "The infrastructure is hosted in a machine room that has all of the necessary bells and whistles, such as cooling, UPS, and generator power," Corey explained. With only 4 racks in the current lab, a decision was made to expand. "We are in the midst of remodeling a 3000 sq ft data center with redundant UPS, generator, HVAC and redundant switching equipment," Scott said. "This facility will be 100 racks large with a focus on density and most of that space will be dedicated to the OSL." He noted that the facility will also provide redundancy for the existing Information Services equipment, which includes the University's core data and communications systems. Scott noted, "we'll open the doors to that facility on 10/1/2005."

The OSL is currently staffed by 10 full-time employees and students. Scott added, "we are about to hire number eleven and twelve ... so if you know any good J2EE developers that want to work with an amazing team, send them our way." The number of students involved is also increasing, "it is a great opportunity for students to get to work with these projects hosted here. They get a fantastic foundation from their coursework and really get to dig in and 'learn' by interacting with this community here. Its a fascinating opportunity for our students and we're lucky to get to have them."

OSL's Future
I asked Scott about OSL's plans for the future, and he replied, "more and more about the community. Our future is two-fold." He explained:

"On the hosting side, we want to give projects an option they wouldn't otherwise have. Let's face it, open source is gaining traction everywhere. With its success comes additional drains on already strapped resources. We want to be an option for these open source projects that don't want to be indentured servants of the big companies that want to provide help. We're not anti-company or anti-people-making-money-from-open-source; we just know that projects want to ensure the future of their communities and so are very careful about who they partner with. We know this will take time and as with everything in open source; it's all about the relationships we develop.

"On the development front, we see ourselves helping promote and develop new software for the public sector. We are already involved in several open source projects here at OSU and are about to announce a Public Sector open source conference to be held in September in Portland. We are hoping to help our brethren at the State level 'get to open source' the same way we have.

KernelTrap's New Home
The server owned by KernelTrap is currently in California, while the Open Source Lab is in Corvallis, Oregon. Thus, to prevent downtime while the server is shipped to my home in Florida for an overhaul, the OSL generously provided an intermediate server dedicated to KernelTrap. The temporary server is a beefy Dell PowerEdge 2850 housing Dual 2.8GHz Xeons, with 3 gigabytes of RAM and a 200 gigabyte SCSI RAID. Corey noted, "it's running Gentoo Linux with a 2.6 kernel, so the software is built & optimized for the Pentium4/Xeon architecture."

A permanent enhancement that affects KernelTrap is that the MySQL database is now on a separate server managed by OSL. The MySQL server has similar specs, though with more RAM and less disk space. Corey explains, "our database environment is used by quite a few community projects, including Arklinux, Gentoo Linux, our Bouncer app, and now Kerneltrap.org!" Looking to the future, he added, "we are currently working on a new db server to add to this one in a redundant configuration. We have a pretty high standard for production services. We shoot for the highest uptime we can get, and work to make sure our community sites get the best uptime they can."

Thanks to donations from the KernelTrap community (see the "Server Upgrade" block in the left column of this page), our web server is going to receive larger SCSI drives, more RAM, and possibly a faster CPU. The existing server currently has two 18GB U160 10K SCSI drivers that are mirrored, 1 GB of RAM, and a 2.66 Ghz P4. The goal is to replace the existing drives with 70 GB SCSI drives (already purchased), double the RAM, and possibly increasing the CPU speed. While the upgrade is not strictly necessary at this time, it seemed silly to waste the opportunity while OSL is kindly providing an alternative server. Each month, KernelTrap's user base has continued to grow, and this upgrade will allow it to comfortably continue to do so.

Seems like...

Kim K. (not verified)
on
May 6, 2005 - 10:26am

...Gentoo has a new neighbour.

Thanks Jeremy

Laszlo (not verified)
on
May 6, 2005 - 3:05pm

Just wanted to say thanks again Jeremy for allowing my company to help what you're building here at KT.

Best of luck with OSL, I've heard great things about them.

Stay in touch.

mailing list archives not restored yet

Anonymous (not verified)
on
May 6, 2005 - 3:10pm

I just wanted to point out that the mailing list archives - In particular, I'm thinking about the git ML archive which I used to look at every few days - have not been restored yet. The last archived message seems to be on may 4, i.e. just before the server was moved.

It'd be great if you guys could look into this. Thanks a lot for doing this great website !

re: mailing list archives

Jeremy
on
May 6, 2005 - 3:17pm

Sorry for that. It's next on my todo list, but likely I won't get to it until Sunday...

I went ahead and manually synced the git mailing list for now, so at least you can catch up to this point... :)

Thanks a lot ! Reading now :)

Anonymous (not verified)
on
May 6, 2005 - 4:50pm

Thanks a lot ! Reading now :)

mailing lists

Jeremy
on
May 9, 2005 - 6:40am

I've begun to re-enable the mailing lists. Within the next 24 hours, all of them should be auto-updating again.

At this time, git and lkml are both up-to-date and auto-syncing.

Retreat?

Anonymous (not verified)
on
May 7, 2005 - 6:26am

Looks like this is the latest of a slew of closings and consolidations in the linux/OSS world. Could it be that the whole linux/OSS fad is now in decline? It certainly looks so.

What?

Anonymous (not verified)
on
May 7, 2005 - 5:46pm

Only in the sense that several projects have moved to the same host, one that's gone out of its way to offer its resources to the FOSS community. None of the projects it hosts have closed or consolidated with another. (Did you even read the fine article?)

Wow, thats profound

Anonymous (not verified)
on
May 8, 2005 - 1:09am

Man, I gotta say; Yer smart!

lol

My thanks to the OSL people. It's such an honour to have such great people as a part of the Free Software community.

voidengineer

Wishful Thinking

eno (not verified)
on
May 8, 2005 - 10:42am

Maybe if I ignore OSS it will go away and I won't have to upgrade my skill set. Fat chance. OSS continues to let developers build on the shoulders of those who came before them - resulting in ever increasing momentum.

Proprietary software companies continue to lock away their solutions - forcing each isolated developer to reinvent the wheel. Indeed, forcing developers to leave behind their own work products when they change companies. The foundation of OSS software is unstoppable at this point in time - it runs the Internet and companies like Google, Amazon, Yahoo, E-bay and IBM.

There is no such simple assurance for proprietary solutions, languages or companies. OSS has a solid beachhead and momentum as it colonizes the proprietary ecosystem. It is really past time to begin hedging your bets and learning the new paradigm.

I'm sorry, I didn't see the c

Anonymous (not verified)
on
May 8, 2005 - 10:56am

I'm sorry, I didn't see the closing or consolidation here. All I saw was one set of servers being relocated. In fact, the article mentions the OSL's EXPANSION of their offerings and servers.

Little help with drupal

crackerjackmack...
on
May 7, 2005 - 8:54am

I've been dealing with mySQL professionally for a good while now, as well as drupal. My departments internal website is drupal and housed on seperate servers. I found your greatest performance improvement when using seperate web and Mysql hosts is to use pconnect as it reduces the TCP connect overhead.

You will have one connection per child process of apache. If you do not kill the apache processes, then the connections are always open. Caching, striping, more ram, none of that made a significant differnce when compared to using pconnect.

I know there is a lot of "bad joojoo" about using pconnect on the internet, but I promise, when using always-alive apache children and a mysql server across TCP, its worth it more than any amount of RAM you put into a server. Granted, my internal website doesn't have near as many hits as kerneltrap, but I will say its not idle either. If the Sql server will remain on a seperate server from the webserver, then I highly encourage using that. If you have any drupal or mySQL questions you can always email me :)

Congrats on the OSL hosting, I hope it is to your standards :-D

update

asdfjs (not verified)
on
May 7, 2005 - 5:30pm

Jeremy,

this node in the FAQ could use a little updating now.

inside the OSL machine room

Eric Lease Morgan (not verified)
on
May 8, 2005 - 7:06am

I've been to the OSL machine room as a part of a tour for Project OCKHAM. It was a very nice machine room, and I have a couple of pictures you may (or may not) find interesting. See the cool wiring or the communications closet.

cool, so if a tornado hits th

Anonymous (not verified)
on
May 9, 2005 - 6:26am

cool, so if a tornado hits this data center we have no more gentoo! ;)

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.