This blog post was written by Pam Derringer, news writer for SearchEnterpriseLinux.com.

Last week, lots of IT guys from New York’s biggest banks and stock brokerages took a day off to attend the sixth annual Linux/Open Source on Wall Street conference at the Roosevelt Hotel in the heart of Manhattan.

All ears they were, but suddenly attendees turned shy when the lectures ended and they were asked to share their own views on Linux. Surely, the firms’ PR police or legal watchdogs would find them out and ruin their prospects for career advancement. Promised a mask of anonymity, however, a few attendees opined on the show, and here are their thoughts.

Lack of management tools cited. “I’m a strong Linux advocate,” said one enthusiastic IT manager. “It’s free, open and secure. And if we find issues, we’re able to fix them.”

Five or six years ago, his firm was one of the first in financial services to introduce Linux servers to data centers. Now about 30% or 40% of its machines run on Linux, with most of the remainder running Windows. The firm’s direction, he said, is definitely off Unix and Solaris and onto Linux.

And he couldn’t be more pleased that Red Hat Inc., in turn, incorporated his team’s enhancements, such as changing storage allocations without a reboot into future versions of the operating system. This way, all Red Hat customers benefit and his staff doesn’t have to maintain the improvement separately with every future fix or upgrade.

He is also concerned about improving data center energy efficiency and has explored various options, to reduce energy consumption, including CPUs, memory and lower wattage.

He has also researched the stateless, single-image data center that can be booted up all at once. “Management would be much better,” he said. “We’d only have one operating system image to manage.”

What is Linux’ most telling shortcoming? “Enterprise-class management tools,” he answered, not unpredictably. “But the good news is: Linux is getting there.”

Rising support costs lamented. Another anonymous big-gun attendee said that for about six years his firm has used Linux — mainly Red Hat — on everything from mainframes to blades and servers.

“Linux is getting a faster, better infrastructure,” he said. “But if these vendors want to remain a viable solution, they need to remain competitive with other data center providers. They’re getting like everyone else, adding more middleware and getting more expensive. It’s getting so that the support and maintenance are costing more than the servers themselves. We need to drive competitiveness back.”

More third-party software urged. A third attendee said the main problem with Linux is the lack of third-party software and inadequate vendor support. For five or six years, he has used Linux to run Web applications and noted that the third-party software shortage is less severe for Web apps than for migrations off AIX or Solaris, for example, simply because of higher volume.

The good news is, he said, that vendor support is on the upswing, citing the presence of Oracle and IBM at the trade show.

“The demand for Linux is there but the growth of third-party software products is slower,” he said. “But we will start to see this [third-party software] materialize more and more.”

Log files may be the most important piece of forensic information we have when determining why a server or application crashes. However, warnings of such a distaster are available to IT administrators. They just have to know where to look (hint: what do you think log files are for?)

Looking for a repeating pattern in a list one thousand items long might seem daunting, but luckily there is help. There’s no need to fear, Splunk is here.

Splunk is an amazing little web application (currently at version 3.1.3) that indexes just about any type of log file you can think of. Not only does Splunk index the information, but it presents it as a beautiful, easy-to-use, web application (purists need not worry, you can access the information from a terminal as well.) So you say, what is the big deal about searching log files? You say that you can do that with grep. That is true, but Splunk is hundreds of times more powerful and excels in four areas:

• Indexing
• Presentation
• Analysis
• Collaboration

Indexing

Splunk can index logs from a number of sources:

• Files and directories
• FIFO queues (pipes)
• Network ports (syslogging directly to Splunk)

Splunk enables you to tail log files, the contents of entire directories, pipes, and even open ports for applications to send their logs directly to Splunk itself (although I recommend using a separate syslog server in order to maintain a file-based log rotation history.)

Presentation

Also, Splunk is more physically appealing than grep (no offense, grep). To give you an idea of what data looks like in Splunk take a gander at this screenshot:

Looking at log files has never been so much fun!

Analysis

This is where Splunk really outshines its command line competition. Imagine you wanted to comb your log files to figure out which VM has had the most number of VMotion events in your VMware Infrastructure? With Splunk that is as easy as pie — a pie chart, that is:

Splunk allows you to easily query the data using SQL in order to build complex analysis reports. And if that was not enough…

Collaboration

Splunk not only allows administrators to easily determine the goings-on of their servers through log file analysis, Splunk also allows administrators to share their logs with the rest of the Splunk community. Imagine this scenario: a major website’s web servers are crashing and the website’s administrators cannot figure out why. As an interner business, their primary point-of-sale is the web; so if their web servers go offline that is very bad. The administrators are pulling out their hair trying to figure out the problem when one of them realizes they haven’t checked Splunk. Because the administrators at Amazon are participating in SplunkBase they can analyze not only their log files but also the logs of anyone else who uploads logs to Splunk’s community. Bingo! They discover that the problem was a lock that was not getting destroyed.

By themselves, the administrators did not have a large enough data set to determine the problem, but because others had generated similar logs and figured out the problem already, the website admins were able to quickly resolve the issue.

Splunk-tastic!

I’ll say it again, Splunk is great. Apart from VMware Server, Splunk may be my favorite server application to come along in the past few years. I cannot imagine running an enterprise data center without Splunk. See you on SplunkBase!

There’s piling on, and then there’s what happened to the hapless SCO today. Just to recap, SCO basically lost its years-long case against Novell and the Linux operating system earlier this month and then just this week announced it would enter into chapter 11 bankruptcy protection.

And today? SCO fired off a press release announcing it received a notice from The Nasdaq Stock Market indicating that the company’s securities will be de-listed from Nasdaq on September 27, 2007, pending an appeal.

The bare bones press release:

The Nasdaq Staff Determination Letter received on September 18, 2007 indicated that as a result of the Company’s having filed for protection under Chapter 11 of the U.S. Bankruptcy Code, the Nasdaq Staff has determined, using its discretionary authority under Nasdaq Marketplace Rules 4300 and IM-4300, that the Company’s securities will be delisted from The Nasdaq Stock Market and that trading in the Company’s common stock will be suspended unless the Company requests a hearing to review the determination. Pursuant to Nasdaq Marketplace Rule 4804(b), the Company is making a public announcement disclosing receipt of the letter.

The suspension of the Company’s common stock is currently set to occur at the opening of business on September 27, 2007. However, an appeal will stay the suspension of the trading of the Company’s securities pending a panel decision by a Nasdaq Listing Qualifications Panel. The Company intends to request a hearing to review the determination. There can be no assurance that the panel will grant the Company’s request for continued listing.

As this now infamous case (and it’s getting a bit tiresome too, no?) begins to wind down with a series of weekly sputters, coughs and stumbles, I can’t help but think of how irrelevant it’s all become to our audience here at the Log and on SearchEnterpriseLinux.com. So, barring any unforeseen apocalyptic events stemming from the appeals process that’s sure to come, I will not be reporting much on SCO anymore.

Good riddance? Not really. Just irrelevant.

The classic UNIX magic poster by Overacre was distributed at a USENIX conference and featured a wizard with UNIX related signs, portents and doo-dads around him. I wonder what a Linux wizard keeps in his hat these days? And, would the Linux wizard be shown totally owning the Unix one and stealing his market share? I wonder…

Last Friday’s ruling by Federal District Court Judge Dale Kimball *appears* to have been that first step in setting about the end to one of the longer running dramas in the software industry: The now infamous SCO vs. Linux trial. Or, more accurately and specifically, the end of the legal cases initiated by SCO based on claims of UNIX ownership and Unixware copyrights.

Watching and reading all of the commentary from this week, and witnessing the almost meteoric unraveling of SCO’s stock price, I am reminded of a game from my childhood that puts the entire process in a much more understandable — and humorous — light. Mouse Trap. The past four years (this all began in 2003, can you believe it?) have been nothing but the setting up of a series of various trials and motions — much like Mouse Trap’s rickety stairs and bathtub bowling ball — that have built and built the tension year over year until finally that plastic man jumped and tripped the trap that fell onto the unsuspecting mouse. I mean onto SCO. I’m mixing things up here, forgive me.

Or maybe it was a suspecting mouse, depending on who you talk to. Court filings suggest SCO knew all along that its claims were, well, dead wrong. Hit that link to read more on that.

Analyst Charles King, founder of Hayward, Calif.-based Pund-IT Research, said in a research note released today that Judge Kimball’s ruling, which firmly supported Novell’s ownership of the UNIX and Unix-Ware copyrights, “quite simply eviscerated SCO’s hopes and dreams.” I’m inclined to agree. The trap is sprung, the cage is down, and the mouse is caught. Game over. There are no little pieces of cardboard cheese left with which to barter, plead or use as an attack. I think Pamela Jones has most of them now, but I’m sure that Novell and IBM have a few pieces too. SCO’s stock reflects this and has bottomed out at a few dimes or so in value per share.

King notes that SCO posted “a brave statement claiming that the court determined or did not dismiss a number of technical points in its favor.” Keeping with my mousy analysis, it was at this point in my reading that I imagined the mouse sliding a tin cup across the bars of his new cage, making a ruckus to distract observers from the fact that he was helplessly locked away. The fact that a mouse would know how to do that would be distracting enough to begin with, so we’re talking about being really distracted here.

And speaking of Groklaw’s Pamela Jones… she also dissected the SCO letter and her post positively drips with Schadenfreude. Can you blame her though? As I loosely followed this trial for the past three years I’ve observed SCO using almost as much elbow grease in fighting IBM and Novell as it did when it tried to discredit Jones and Groklaw.

Jones seems to think there might be one last breath in SCO’s legal/business team, but in the same breath she doesn’t sound too impressed with what they’ll come up with. King agreed:

SCO’s plans to “continue to explore our options” sounded as hollowly optimistic as a conventioneer in Las Vegas who, upon emerging from the casino where he has squandered his life savings, declares how lucky he is that the pit boss let him keep his Fruit of the Looms and Rotary pin.

Not only did Judge Kimball favor Novell concerning the copyright issues, he also granted the company the right to direct SCO to waive its claims against IBM and Sequent and stated that SCO is obligated to recognize that waiver. Kimball also found that SCO is obligated to pay Novell for license fees the company collected from Sun and Microsoft in 2003. As much as 95% of those fees, which total some $18.3 million, could be due to Novell. However, since SCO has only a fraction of that amount on hand, it remains to be seen how much Novell will ever collect.

And though SCO will still be with us for the time being, King said the case seems close enough to a resolution that it’s worth considering a few lessons learned:

• First, when heading downhill, use the brake pedal instead of the gas. SCO’s situation illuminated a notable issue in the rise of Linux; that it has impacted UNIX solutions far more seriously than Microsoft, a bugaboo of Linux and Open Source aficionados. The result? Like other dinosaurs, UNIX specialists such as SCO were hammered worst by simple market evolution. By pursuing what turned out to be empty claims against Novell and IBM, the company merely hastened its demise.
• Second, when you hope or plan to do business with people, don’t try to fool them. SCO’s aggressive pursuit of IBM (which enjoys notable good will in the Open Source community) and claims that Linux violated UNIC copyrights made the company’s attempts to position itself as an emerging Linux vendor a tough sell, at best. The Linux community enjoys many well-educated, highly skeptical members who do not take kindly to empty braggadocio. SCO’s stick and carrot marketing approach worked in a few specific instances but was destined to fail among a disparate, diffused, and informed community.
• Finally, when heading toward a concrete wall, avoid high speed collisions at all costs. In essence, SCO chose to pursue a high stakes strategy which was finally unsubstantiated by facts. That suggests that the company was deliriously optimistic, deeply misinformed, or simply attempting to bluff its way into the winner’s circle. This approach can succeed sometimes, but not against companies such as IBM and Novell that possess the legal and financial resources to fight a battle to its ultimate conclusion.

Let’s review.

“Without the possibility of a legal pay day, how long will SCO be able to function as a commercial company? If it is headed for destruction, is a partnership or acquisition in SCO’s future, and if so, by who? How will SCO cope with owing potentially millions in licenses fees to Novell, and how far will Novell go in pursuing payment? Will Novell follow Judge Kimball’s lead and order SCO to waive its legal claims against IBM? Will the resolution of these legal cases have any tangible impact on Linux of the Open Source movement? As the SCO melodrama winds toward its sad if predictable dénouement, these are points we will be watching carefully,” King said. All good questions, and they’ll probably all have answers sooner rather than later.

On that note however, it sounds like we’re about ready to play a new game with SCO. Any ideas out there? Oh, I know what we could play. How about Sorry!?

There wasn’t as much clapping as I thought there’d be. What I mean is; is when a user stepped up to the microphone to grill Novell CEO Ron Hovsepian about Microsoft, the GPLv3 and those infamous “coupons,” there wasn’t the zealous explosion of applause that I had become accustomed to over the years.

There was some, to be sure, but the response was most definitely more subdued than I thought it would be when Hovsepian opened his keynote with the revelation that he’d be taking questions from the packed hall. In fact, I at first thought maybe he’d have donned various pieces of plate mail as he spoke at length about Linux’s rise to dominance in the data center. This never came to be, and the navy sport coat remained in place; apparently all this executive needed to deflect the four — yes four — questions from a room that held hundreds of sitting and standing Linux geeks.

Anyway, the Q&A…

Q -With so many distros, all with multiple interfaces and APIs specific to each distro, how will you get your first point about standardizing ISVs done?

Ron Hovsepian: That is our tech challenge; we need to leverage the bodies that exist already. We don’t need to create new ones, but we need to take advantage of what the Linux Standard Base has already created. Most of the parts are here but the earnest effort to work with ISVs to get them to standardize on what they are comfortable with remains.

Q – Are you talking to other stakeholders inside of the distro side of things, as well as components?

A- We are having real time conversations now with different partners, and obviously this will be a vendor neutral application driven by people in this room. It is a critical success factor for entire market

Q- You stated that Novell will ship GPL3? Then how will Novell support MS coupons and customer s with GPL3 software when Microsoft has explicitly said that they will not support or endorse GPL3? (pause for applause — Jack)
A – Fundamentally what this relationship is, is that as a customer goes to redeem certificates the customer calls up Novell and redeems, and then with that certificate we will give them the most current release we are shipping to the market — including GPLv3 code. What Microsoft feels is they are not a legal party to the contract. In its simplest form it is a coupon that customers redeem and we’ll deliver to them latest distro on the shelf at that time.

Q – What is the official Novell position on OpenDocument?
A – Life is a series of steps. Novell’s position is that we absolutely want the OpenDocument format (ODF). That is our primary sort key. We support the Open XML translator, and we will talk with whatever group we can get to get things done. We must separate the religious argument and take whatever steps we can take to get customers over to the ODF approach. We support ODF’ that is first sort criteria. We also support OpenXML translators to get you to ODF over time.

More to come!

“Apple buys CUPS.”

It was a fun little news item to mull over and chew last week, but what’s the precedent? What’s the big picture when it comes to big name corporate entities snatching up open source projects or their core developers? It’s happened, and will happen again, but is it increasing in frequency? Staying the same? A non-issue?

When Apple bought up CUPS last week, some industry watchers were surprised, but not all. First, Apple’s been using CUPS since 2002. Second, Raven Zachary, senior analyst with The 451 Group, said this kind of acquisition — big name vendor snapping up renown open source project and core developers — is happening more and more. And the trend is set to to continue at an even quicker pace than before.
But before we get into that, a bit of background: CUPS, for those not in the know, is the Common UNIX Printing System. According to the shifting sands of crowd mentality over at Wikipedia.org, CUPS is a modular printing system for Unix-like computer operating systems that allows a computer to act as a powerful print server. A computer running CUPS is a host which can accept print jobs from client computers, process them, and send them to the appropriate printer. Sounds like fun.

Zachary himself has first-hand experience with a corporation snapping up a core developer. In fact, while he was employed at La Quinta, his company and Goldman Sachs almost came to corporate fisticuffs over a lead Apache Tomcat developer. “We ultimately were successful in bringing this developer on-board,” Zachary wrote over at the CAOS Theory blog, “but the reasoning had more to do with locale and the cost of living (Manhattan vs. Dallas) than it did with the salary and deal terms.”

Michael Sweet, the creator of CUPS (Common UNIX Printing System), announced last week that Apple acquired the rights to CUPS in February 2007 and that he is now employed full-time by Apple. He’s the kind of developer that Zachary is talking about. Even though Apple is now the copyright holder, CUPS will continue to be made available under the existing open source licenses (GPLv2 and LGPLv2).

It’s now after the fact however, so what’s going to happen to CUPS? Today CUPS is available under the GPL, but will Apple restrict access to it over time? Zachary doesn’t think so, and lays out a pretty good case for openness.

The strength of CUPS is the large community out there writing CUPS drivers for printers. If I were writing CUPS drivers and Apple starting restricting licensing terms, I would stop writing drivers (or advocate forking the project). Apple wins when there are more CUPS printer drivers out there. That said, Apple’s reputation in the open source community is not spotless. There is some concern out there regarding the future of CUPS, understandably, but Apple is not likely to give much more information out other than it has no plans to make any changes to the CUPS licensing terms.

This process, according to Zachary, is set to reproduce itself time and time again in the near future, and with great frequency too. There’s a little wiggle room in there as to what the buyers are going to do with the code they snatch up, but there’s a pretty good case for openness in there too. What’s next? Why? We’ll know soon enough, right?

For those of you local to the Metro NY area, you may want to consider Unigroup as a user group hangout. It’s one of the best user groups out there on Unix and Linux in NYC. I’ve been a member for quite sometime and actually was a board member awhile back. It is a very technical group, so be prepared for heavy stuff. Sorry about the late notice, but there is a meeting tomorrow night in NYC. Here are the details and the agenda:

When: THURSDAY, June 21st, 2007 (3rd Thursday)
Where: Alliance for Downtown NY Conference Facility
Downtown Center
104 Washington Street
South West Corner of Wall Street Area
Downtown, New York City

** Please RSVP (not mandatory) **

Time: 6:15 PM - 6:25 PM Registration
6:25 PM - 6:45 PM Ask the Wizard, Questions, Answers and Current Events
6:45 PM - 7:00 PM Unigroup Business and Announcements
7:00 PM - 9:30 PM Main Presentation

Topic: UBUNTU Linux

Speaker: Nathan Eckenrode,
New York Ubuntu Local Community (LoCo) Team

Meeting Outline:

- What is Ubuntu Linux?
- Ubuntu Latest Version: 7.04 Desktop and Server Editions.
- What does Ubuntu look like?
- What products does it include?
- What sorts of special effects does it run?
- What sort of support can a new User receive?
- What about the Dell deal?
- What is in the works for Gutsy Gibbon?
- What sorts of issues when a new User switches to Ubuntu Linux?
- Can Ubuntu Linux work in my Environment?
- About the New York Ubuntu Community.
- What is the New York City Ubucon?

PS. I have to admit, this is my first cut at a blog, so I hope that I will like doing this and more importantly, that you will like what I have to say. In this blog, I’ll be talking about infrastructure technology, with a focus on Unix, Linux, databases and networking. I look forward to your comments and suggestions as to how to make this a better place to hang out.