In a conversation that began as a request to include the SAS Transport Layer in the mainline Linux kernel, there was an interesting thread regarding specifications. Linux creator Linus Torvalds began the discussion saying, "a 'spec' is close to useless. I have _never_ seen a spec that was both big enough to be useful _and_ accurate. And I have seen _lots_ of total crap work that was based on specs. It's _the_ single worst way to write software, because it by definition means that the software was written to match theory, not reality."
Linus went on to list two reasons to avoid specifications when writing software. First, "they're dangerously wrong. Reality is different, and anybody who thinks specs matter over reality should get out of kernel programming NOW." Second, "specs have an inevitable tendency to try to introduce abstractions levels and wording and documentation policies that make sense for a written spec. Trying to implement actual code off the spec leads to the code looking and working like CRAP." As a "classic example" he pointed to the OSI model, "we still talk about the seven layers model, because it's a convenient model for _discussion_, but that has absolutely zero to do with any real-life software engineering. In other words, it's a way to _talk_ about things, not to implement them. And that's important. Specs are a basis for _talking_about_ things. But they are _not_ a basis for implementing software."
From: Linus Torvalds [email blocked] To: Arjan van de Ven [email blocked] Subject: Re: I request inclusion of SAS Transport Layer and AIC-94xx into the kernel Date: Thu, 29 Sep 2005 12:57:05 -0700 (PDT) On Thu, 29 Sep 2005, Arjan van de Ven wrote: > > a spec describes how the hw works... how we do the sw piece is up to > us ;) How we do the SW is indeed up to us, but I want to step in on your first point. Again. A "spec" is close to useless. I have _never_ seen a spec that was both big enough to be useful _and_ accurate. And I have seen _lots_ of total crap work that was based on specs. It's _the_ single worst way to write software, because it by definition means that the software was written to match theory, not reality. So there's two MAJOR reasons to avoid specs: - they're dangerously wrong. Reality is different, and anybody who thinks specs matter over reality should get out of kernel programming NOW. When reality and specs clash, the spec has zero meaning. Zilch. Nada. None. It's like real science: if you have a theory that doesn't match experiments, it doesn't matter _how_ much you like that theory. It's wrong. You can use it as an approximation, but you MUST keep in mind that it's an approximation. - specs have an inevitably tendency to try to introduce abstractions levels and wording and documentation policies that make sense for a written spec. Trying to implement actual code off the spec leads to the code looking and working like CRAP. The classic example of this is the OSI network model protocols. Classic spec-design, which had absolutely _zero_ relevance for the real world. We still talk about the seven layers model, because it's a convenient model for _discussion_, but that has absolutely zero to do with any real-life software engineering. In other words, it's a way to _talk_ about things, not to implement them. And that's important. Specs are a basis for _talking_about_ things. But they are _not_ a basis for implementing software. So please don't bother talking about specs. Real standards grow up _despite_ specs, not thanks to them. Linus
From: Luben Tuikov [email blocked] Subject: Re: I request inclusion of SAS Transport Layer and AIC-94xx into the kernel Date: Thu, 29 Sep 2005 16:20:13 -0700 (PDT) --- Linus Torvalds [email blocked] wrote: > > A "spec" is close to useless. I have _never_ seen a spec that was both big > enough to be useful _and_ accurate. > > And I have seen _lots_ of total crap work that was based on specs. It's > _the_ single worst way to write software, because it by definition means > that the software was written to match theory, not reality. A spec defines how a protocol works and behaves. All SCSI specs are currently very layered and defined by FSMs. This is _the reason_ I can plug in an Adaptec SAS host adapter to Vitesse Expander which has a Seagate SAS disk attached to phy X... And guess what? They interoperate and communicate with each other. Why? Because at each layer (physical/link/phy/etc) each one of them follow the FSMs defined in the, guess where, SAS spec. If you take a SAS/SATA/FC/etc course, they _show you_ a link trace and then _show_ you how all of it is defined by the FSM specs, and make you follow the FSMs. > So there's two MAJOR reasons to avoid specs: Ok, then I accept that you and James Bottomley and Christoph are _right_, and I'm wrong. I see we differ in ideology. > It's like real science: if you have a theory that doesn't match > experiments, it doesn't matter _how_ much you like that theory. It's > wrong. You can use it as an approximation, but you MUST keep in mind > that it's an approximation. But this is _the_ definition of a theory. No one is arguing that a theory is not an approximation to observed behaviour. What you have here is interoperability. Only possible because different vendors follow the same spec(s). > - specs have an inevitably tendency to try to introduce abstractions > levels and wording and documentation policies that make sense for a > written spec. Trying to implement actual code off the spec leads to the > code looking and working like CRAP. Ok, I give up: I'm wrong and you and James B are right. > The classic example of this is the OSI network model protocols. Classic Yes, it is a _classic_ example and OSI is _very_ old. _But_ the tendency of representing things in a _layered_, object oriented design has persisted. > spec-design, which had absolutely _zero_ relevance for the real world. > We still talk about the seven layers model, because it's a convenient > model for _discussion_, but that has absolutely zero to do with any > real-life software engineering. In other words, it's a way to _talk_ > about things, not to implement them. Ok. > And that's important. Specs are a basis for _talking_about_ things. But > they are _not_ a basis for implementing software. Ok. Let's forget about maintenance and adding _new_ functionality. > So please don't bother talking about specs. Real standards grow up > _despite_ specs, not thanks to them. Yes, you're right. Linus is always right. Now to things more pertinent, which I'm sure people are interested in: Jeff has been appointed to the role of integrating the SAS code with the Linux SCSI _model_, with James Bottomley's "transport attributes". So you can expect more patches from him. Regards, Luben P.S. I have to get this 8139too.c network card here working.
From: Linus Torvalds [email blocked] Subject: Re: I request inclusion of SAS Transport Layer and AIC-94xx into the kernel Date: Thu, 29 Sep 2005 17:35:27 -0700 (PDT) On Thu, 29 Sep 2005, Luben Tuikov wrote: > > > It's like real science: if you have a theory that doesn't match > > experiments, it doesn't matter _how_ much you like that theory. It's > > wrong. You can use it as an approximation, but you MUST keep in mind > > that it's an approximation. > > But this is _the_ definition of a theory. No one is arguing that > a theory is not an approximation to observed behaviour. No. A scientific theory is an approximation of observed behaviour WITH NO KNOWN HOLES. Once there are known holes in the theory, it's not a scientific theory. At best it's an approximation, but quite possibly it's just plain wrong. And that's my point. Specs are not only almost invariably badly written, they also never actually match reality. At which point at _best_ it's just an approximation. At worst, it's much worse. At worst, it causes people to ignore reality, and then it becomes religion. And that's way _way_ too common. People who ignore reality are sadly not at all unusual. "But the spec says ..." is pretty much always a sign of somebody who has just blocked out the fact that some device doesn't. So don't talk about specs. Talk about working code that is _readable_ and _works_. There's an absolutely mindbogglingly huge difference between the two. Linus
From: Theodore Ts'o [email blocked] Subject: Re: I request inclusion of SAS Transport Layer and AIC-94xx into the kernel Date: Fri, 30 Sep 2005 01:31:49 -0400 On Thu, Sep 29, 2005 at 04:20:13PM -0700, Luben Tuikov wrote: > > A spec defines how a protocol works and behaves. All SCSI specs > are currently very layered and defined by FSMs. A spec defines how a protocol works and behaves --- *if* it is well-specified and unambiguous, and *if* vendors actually implement the spec correctly. (And sometimes vendors have major economic incentives to cheat and either intentionally violate the specification, or simply not bother to test to make sure whether or not they implemented their hardware correctly.) Computing history has been literred with specifications that were incompentently written and/or incompentently implemented --- from the disaster known as ACPI, to FDDI (early FDDI networking gear was interoperable only if you bought all of your gear from one vendor, natch), consumer-grade disks which lied about when data had been safely written to iron oxide to garner better Winbench scores, and many, many, many others. This is one of the reasons why the IETF doesn't bless a networking standard until there are multiple independent, interoperable implementations --- and even _then_ there will be edge cases that won't be caught until much, much later. In those cases, if you implement something which is religiously adherent to the specification, and it doesn't interoperate with the real world (i.e., everybody else, or some large part of the industry) --- do you claim that you are right because you are following the specification, and everyone else in the world is wrong? Or do you adapt to reality? People who are too in love with specifications so that they are not willing to be flexible will generally not be able to achieve complete interoperability. This is the reason for the IETF Maxim --- be conservative in what you send, liberal in what you will accept. And it's why interoperability testing and reference implementations are critical. But it's also important to remember when there is a reference implementation, or pseudo-code in the specification, it's not the only way you can implement things. Very often, as Linus has pointed out, there are reasons why the pseudo-code in the specification is wholely inappropriate for a particular implementation. But that's OK; the implementation can use a different implementastion, as long as the result is interoperable. Regards, - Ted
Call me crazy, but a scientif
Call me crazy, but a scientific theory is only what is proported to be the best explaination to a set of natural observed occurances and behaviors. It isn't required to be without holes. Perhaps Linus is referrign to Laws, which are not certified by anybody, but assumed to carry the weight of universal (or at least near universal) truth, such as the Law of Gravity or the Law of Conservation of Mass and Energy.
Certainly Linus is allowed to say that a particular spec is useless, but I'm not throwing out my processor spec book anytime soon. The point is that useful specs match reality. It would have been nice if this post had included anything reguarding whether the protocol in question actually matches reality or not.
Linus' hole in a theory seems
Linus' hole in a theory seems to be a counter case for the theory, NOT an area which isn't explained by the theory.
Keeping that meaning in mind, if a theory has such hole it means that there exist something which proves that the theory isn't correct, in which case the theory isn't valid.
If my theory is that all cats are green, and someone encounters a black one, it means my theory is bogus. Perhaps most cats are green, in which case the theory becomes only an approximation (but in this case even that's not true ;-).
There's no real difference between Laws and theories, except that laws maybe were more deducted from experimental data instead of thought out theoretical theories. "This seems to be always true, but we don't know why".
Very close. Theories usually
Very close. Theories usually attempt to explain why or how things happen, while laws are always just generalizations for what does happen. Laws often take the form of mathematical relationships which express what always happens, without attempting to explain why.
Many of our best scientific theories "have holes"
To use a really specific example, Einstein's General Theory of Relativity and Quantum Electrodynamics - two of our best present theories - are in conflict. Stating that they "aren't valid" is a copout against how science works. We just end up using them, and trying to find some way we can actually get at the edge cases that may give us a clue to how to unify them.
Eivind.
Conflict in what way? Do t
Conflict in what way?
Do they predict different things for certain situations? If so, which one is following reality and which one isn't? If they both are close, but not quite yet it, it means they are good approximations of reality. If they conflict logically, but reality seems to follow both then that's only more interesting.
That a theory is invalid doesn't necessarily mean it's useless, it just means reality doesn't agree with it. Quantum mechanics may be more valid than Newton, but that's not what mechanical engineers will use when calculating certain things.
What scientics want are cases which prove that their theory is bogus, so they have a clue for thinking up new ones which hopefully describe everuthing in a non-conflicting and hopefully more elegant way.
Conflict in what way? Quan
Quantum mechanics doesn't have a description of gravity, and general relativity doesn't work on microscopic scales.
So it is difficult to describe the inside of a black hole or the beginning of the universe, because quantum mechanics operates on those scales but does not describe gravity, and general relativity produces absurd answers on those scales.
As yet, nobody has been able to fly inside a black hole or ride a time machine to the big bang in order to observe what really happens.
The problem is that Torvalds
The problem is that Torvalds is talking not about what is a correct scientific theory but what _is_ a scientific theory, suggesting that if they have "holes" they don't constitute science. This is a very naive theory of demarcation, and highly problematic since it would exclude any falsified theories from being science at all. Newton's gravity laws for example. Since we can never know whether a theory has holes of the suggested kind (falsifying holes), because of a well-known problem with inductive reasoning, we can also never know whether a theory is scientific or not. So accordingly, nothing is science.
leave science out, please :)
at the same time, a scientific theory is a mere intellectual exploration space. to compare it to specs is misleading. please leave science out of specs. science doesn't care about specs. if at all, it makes some sense in engineering problems. it does make interoperability easy but only in a perfect world. so i'm leaning towards mr. torvalds :) on the "usefulness" of specs.
i guess, by processor spec book you mean what's already been "implemented" in the hardware. if x86 spec was useful, you should have been able to swap a processor made by Intel with one made by AMD. ofcourse you can atleast compile your programs to a generic x86 spec, but it is only an interoperability "compromise".
my 2 cents :)
you can run i386 binaries on
you can run i386 binaries on all Intel AMD, via ,etc processors thanks to the x86 spec.
Not an argument for Specs at all.
As far as I'm aware x86 was never an open agreed upon standard, people knew what commands where available and what they did, they just reverse engineered the bugger. Then Cyrix had a bunch of extensions, then intel had a bunch of extensions, then they stuck the MMX code on to it, then it became incorporated Then AMD extended it again, etc etc all the way up to AMD64. One big bloody mess.
Proving Linus's last point that standards grow up despite specs not because of them.
Wrong, the Intel x86 arch boo
Wrong, the Intel x86 arch books are FREE and cover every instruction, opcode, etc.
You don't truly believe that do you?
Ever hear of Appendix H? Sure, it's open now but wasn't before. Also, there are so many corners of operation that need to be faithfully emulated between different devices.
Robert Collins made a career of exposing all the undocumented behavior of x86 processors. Check out this page especially.
Specs have problems, but are indispensable
The extensions that Intel & Cyrix did are just that - extensions. The basic instructions have remained the same. Software that limits itself to these runs without modification on Pentium or Athlon. Besides, AMD now supports MMX, so that's also a standard, in the sense that it's available on most processors.
It doesn't matter where a standard is de-facto or formal as much as it does whether one exists at all or not. Without x86 we'd have to recompile our programs everytime Intel or AMD released a new processor.
My position is:
- many (most?) specs are bad.
- a spec should be developed after implementation and not as an a priori thery.
- specs shouldn't be finalized unless there are two independent implementations that have been found to interoperate.
- a formal spec is better than a de-facto one, but the latter is much better than fragmentation.
yeah, but the x86 spec was b
yeah, but the x86 spec was built, then more importantly, changed to match reality once the actual chips were built by Intel, so it's now something that can be implemented, since it's based on reality.
I think Linus's point still stands, you can't design something on paper, then implement it exactly as the paper says to, because at best, what's on the paper is a collection of ideas about how someone thinks something should work. It's based on theory, and suppositions, which are known to be wrong from time to time. This is the best anyone can be expected to do, without the real thing in front of them. Whether scientific, or engineering, not everything we think will work, does. If it did we wouldn't need debuggers or optimization.
You can shoot for that lofty goal, meeting a spec with implementation, but at the end of the day, there is "what should be" and "what is".
They never match in software engineering, or anywhere else for that matter. What matters is what's in the binary, or even what's in the chip. Intel had a blueprint for x86. Along the way it changed. Once they had something that worked they documented it so others could repeat it. They knew what worked when they built this spec, and knew it could be used to build something that reflected this spec in reality because that's what it was based on.
Even Architects, revise and revise as the engineers build a building and what you end up with is an "as-built". Some things which work on paper, simply don't work in reality. A pipe ends up running into a beam, or a duct ends up outside the building because a measurement on the paper is wrong. Anyone that thinks a blueprint or spec will match reality at the end of the day, isn't in it.
I used to be an electrician, before going back to college. I know this to be true. Sometimes the architect wants 1" accuracy on center for a light, but if the light ends up in a beam, it's gotta be moved. No two ways around it. Software has similar issues.
Think about it.
-Anonymous
Documentation
Documenting the code isn't usefull too. It never reflects reality. So, just quit commenting code, writing user/architecture documentation.
Coding
Sure it is. I think the above discussion is on writing code to the documentation, not the other way around as you seem to have it.
wrong target
Linus is an engineer/tech. He dislikes theory work because it often gives nothing in practice.
However, specs are not always theory, and they may be usefull, as well as docs. He may be smart enough (or know linux code enough) to not need any doc/spec, but it's not the case of many other people. Some specs are good, and sometimes necessary.
He cited OSI model, well, but I can assure you I won't go in an airplane if it was done with Linus' practices... There are specs in some places that are good, and that are read and followed. Even in non-dangerous domains such as Web standards, specs are necessary, and those who don't follow these specs make crap softwares/browsers!
Moreover, in Linux development model, which is fuzzy and distributed, not directed, defining the software may be vain. However, in a commercial environment, defining the spec is really writing a contract, which protects both the customer and the editor. Specs there defines what the software can and must do, and ensures it will do. Linus obviously lacks of experience in industrial and critical projects. He may be right for the kernel development (however I still doubt it should be so entire on that subject), but he's wrong on many other domains.
IOW, Linus does here a generalization which is at least as wrong as are the examples he cited. As we say : "all generalization are false".
If he finds a bad spec, either it throws it away, or he fixes it. It's the same for technical docs. But he shouldn't tell every specs are useless and bad. That's wrong.
specs
specs snatched out of thin air just dont work. specs drawn and refined by experimentation are the ones that are useful.
i never write specs when i start a software project. i start with small test programs that do what is required.
right, but limited to small things/implementation details
Knowledge of technology, given through small tests, is a good thing to get a spec. However, you cannot build a whole complex software with limited cost blindly.
If you do not have at least a plan, you won't do what you should. The software may be usefull and correct, but it certainly will be inadequate and don't do what was asked for.
And building specs on top of experience will certainly prevent you from controlling you development process, therefore forbid access to many big and serious projects. I'm sorry, but when things get really complex, hackers have no place. Structure, rigor and organization have, and are far more important than good or bad code. This is were specs appears (among other things of course).
And do not tell me a kernel OS as Linux is a complex thing as an example: it is big (wide?) because of many hardware, but the core OS still is simple (vertically). Moreover, Linux was rewritten several times, showing perfectly that a. nobody told initially what the needs were, and b. it was several times wrong. In an industrial process, this is simply unacceptable (you simply won't have the contract).
linux isn't targeting a singl
linux isn't targeting a single customer nor a single process. it is flexible and always evolving, unlike industrial process where you design something once and you're done with it. there's no comparison between the two, they are completely different domains.
just because linus doesn't have any specs to show you, doesn't mean linus doesn't have a plan. don't confuse the two.
even specs don't help produce reliable code in the industrial process domains. what counts there is good peer review and testing to the limits of your sanity. choosing a good language can help too.
that's wishful thinking, afai
that's wishful thinking, afaiac. in my experience, commercial software is rewritten just about as often as non-commercial software, for one thing. for another, specs often are complete crap. not so much because they contain bad ideas, but more often because they are overly complex or worded in an ambiguous manner.
take soap, for example. overengineered, because when there was no real implementation lots of things probably sounded like good ideas. now half the implementations implement half the spec (all different halves), and have to concentrate on a common subset in order to interoperate. you know what? that common subset is almost as simple as xml-rpc, which was defined because soap was deemed to complex.
that being said, xml-rpc is rather limited because - among other things - it allows only for 32 bit integers. it's not a great spec either, but it works because it's easily implemented and simple enough that interoperability problems are rare.
i don't think linus is right in dismissing specs. some specs, such as rfc 2045/2046 are complex and still pretty unambiguous. you can work with them, therefore they are useful specs.
And how do you know what is r
And how do you know what is required? How do you know a test succeeds or fails? Test goals are necessary. A document listing those test goals could be called a spec (there seems to be some confusion about what a spec contains and how it differes from a list of requirements). A good spec is then a complete and unambigous list of test goals. Such a list might grow over time or might need modification, e.g. if it is found that some goals are not achieveable (in a given timeframe, at a given cost).
wrong target
I have the distinct, yet humble, impression Linus is not talking about NOT complying to specs as a matter of establishing standardisation but as NOT complying to specs when actually writing code wich is to be used by a piece of software wich is dependant on a/the spec.
In the end, the spec is the 'communications layer/protocol', the code is both the carrier and the signal. It doesn not matter on wich 'material' the 'carrier' is built or in wich 'modulation' the 'signal' is sent, as long as it complies with the demands for the spec at some point along the 'transmission' it will work without any problem.
Maybe he could have reffered to this as a "blackbox" programming technique ;-)
Though i'm by no means a programmer of any kind and have no fundamental insights on the linux-kernel i do understand that after all what matters is the final product is supporting the linux system calls, what's in that 'blackbox' does not matter all that much as long it's working up to expactations.
note: even hardware support is being dealt with in the same fashion
I'm not sure any spec can handle this as even specs are subject to interpretation on both sides, and even by the implementation a spec can be interpreted contrary to expectation (the reality vs spec argument). In short anyone should be able to admit a spec does not guarantee any predictable results.
note: This looks like a good step-up for an ITIL discussion as well
To my mind, Linux is also pro
To my mind, Linux is also provocating and he's right. There are many, many examples of standards and specifications that are simply not followed, because they conflict with the real world or are too complicated. I have also experienced this very often.
This does not mean that specs are bad, quite in the contrary they are
to my mind very important but they must be always in contact with the real world - and this is often not the case.
For example - have a look at ISDN: This was specified over and over and in the end every country implemented his own ISDN standard. Have a look at SOAP: 200 pages, extremely complicated, few implementations that are not fully compatible. There are many, many other examples.
Nevertheless all these specifications are important, similar to all those computer languages that died out. People are "taking the best and leaving the rest" - and there is for sure a *lot* of "good" in all these specifications.
Web specifications, for examp
Web specifications, for example XHTML and CSS, are the perfect example of the point Linus is trying to make, and at the same time they prove that specs are a necessity.
Web browsers, such as Internet Explorer and Firefox, supposedly implement the XHTML and CSS specifications. In reality, a web developer often finds himself writing compliant code only to realize it doesn't work the same in any two browsers. If you don't believe this to be true, try creating a pages loaded with block-style entities, lots of floats, margins, padding, and borders, and see what you get.
In _reality_, the spec is an approximation *at best*. If you follow the spec to the letter, your code probably will not interoperate, and while you haven't done anything wrong, it simply will not work.
In _reality_, the spec is the only thing keeping web browsers remotely similar. If the spec didn't exist, things would be far worse.
So, specs are not "useless", but they should never be regarded as absolute. Think of them more as "guidelines" than as "laws".
As for theories and holes, there is a *VERY* big difference between a theory and a scientific theory. A theory is simply " a concept that is not yet verified but that if true would explain certain facts or phenomena"; this is what we call a _scientific hypothesis_. A scientific theory is a "theory that explains scientific observations"; or in other words, there are real observations already suggesting it to be true. And, as far as I can see, Linus didn't specify what a "hole" was. He may have meant "unexplained related observations", or he could have meant "observations which disprove the theory". The theory of evolution suffers from the former, while the theory that "all cats are green" suffers from the latter.
Lastly, it is allowed for theories to conflict, so long as they don't disprove each other. For example, the theory that light is a particle, and the contradicting theory that light is an energy wave. Neither disproves the other.
I think Linus is way out of l
I think Linus is way out of line here.
As an engineer he should know that a spec is as much a work in progress as the code that implements the desired behaviour for a device!
The tendency in technology is to create devices with greater and greater complexity. A way to keep a grip on this complexity is to create specs, have interoperability tests, plug fests, etc.
A released spec is just a point in time at which the spec covers desired behaviour for a device _at that time_. of course there can be mistakes in a spec, as there can be mistakes in an implementation of a device. just like there can be mistakes in the code!
stop being so childish Linus and accept the fact that in the real world specs are in fact used. specs are not bibles or fixed laws but approximations of reality (like you pointed out yourself), just as the code is an approximation of reality!!!
as a side note: maybe you should talk to a few CPU engineers, they will tell you that specs are the only way to go when implementing a CPU (or any other piece of complex hardware). re-spinning the chip in the fab is quite expensive and you will want to avoid this, hence the specs: they define the behaviour and the implementation can be tested against it....
in reality, spec == best guess by a bunch of theorists.
specifications in programming/software are not like specifications in engineering.
Programming/Software contains a lot of art and especially a lot of NEW stuff that hopefully works......certainly very little of the tried and true stuff you find in engineering. And very little testing like in engineering, IF any, BEFORE actually going live.
Programming/Software != engineering
Software doesn't even come close to the rigors of engineering by any means.
You are EXACTLY right!
You are EXACTLY right!
and THAT is EXACTLY why todays software is generally crap.
Once programming software == the rigors of engineering I will finally be out of a testing job.
It can't happen soon enough, and I for one would have preferred Linus to be helping in the drive for this to happen, as opposed to helping perpetuate the situation of software engineering having no more discipline than any other transient hobby activity.
Hardly out of a job, trust me.
A big portion of engineering is devoted to testing, actually. In fact, for sufficiently large projects, it's more than quite possible you'll have more test engineers than developement or research engineers.
The big difference is that the stuff you'll be testing will have behavior that's more completely defined and you'll be going after much deeper, more interesting bugs rather than mundane crap like "Clicking 'Open file' causes it to crash."
Writing code without a spec i
Writing code without a spec is proof of ADD.
say what???
what????
I don't know how you make software but for me it's still called software engineering.....
I wouldn't want to have a car with an ABS system that 'contains a lot of art and especially a lot of NEW stuff that hopefully works'
so maybe you're right if you're just hacking together a little handy tool but for anything that you put on the market I sure as hell hope you're doing some real engineering otherwise I'd like to know what you wrote so I can stear clear of it
and yes, maybe it's about time the SW engineers talked a bit more with the HW engineers!
I can tell you from experience that this would be a Good Thing since HW engineering is much more mature than SW engineering while in practice there isn't a huge difference in the methods they (can) employ (w.r.t. digital HW engineering that is)
Different kinds of software
Certain domains of software do have deep rigor and truly employ software engineering. Half of the crap on your desktop, though, is held together by baling wire and chewing gum.
"I can tell you from experien
"I can tell you from experience that this would be a Good Thing since HW engineering is much more mature than SW engineering while in practice there isn't a huge difference in the methods they (can) employ"
I think everyone is deliberately missing Linus's point, he is saying that specs are useless for coding. Precisely because the hardware isn't engineered to meet the specifications as it should. So don't go telling him the software engineering isn't as mature as the hardware engineering.
There is a lot wrong with software engineering, but few, if any hardware engineers deal with a project of the complexity of the Linux kernel. It is over 22,000 files, comprising when compressed about 26 million characters, it comprises well over a million lines of code, it runs on a fantastically huge configuration of hardware from an unmatched variety of vendors.
The manual effort expended on it has been estimated to be of the orders of billions of dollars. The closest in terms of hardware scale, would be perhaps a powerstation, airliner, or aircraft carrier. Of course it is only one component that has to live in another universe of other (almost equally complex) components.
Software engineering is different.
Linus supplies a lot of specifications to the world - they are called API's, and Linux provides plenty of them. Representing a contract between software and kernel, that it will do the right thing, or return the right error if it can't. The difference here is that the code IS the specification, the code has to be correct to an exacting degree otherwise the stuff that relies on it will malfunction. You can't readily simplify it or generalise it - if something needs a time attached to it, you need to say what time. to what precision, stored in what number format, and representation, and if you get it off even by one bit. That stuff doesn't simplify easily, if it could you might have a specification Linus would like, and you could mechanically turn it into functioning code as well.
I've seen big hardware engineering, and big software engineering, and agree there is a lot more most software engineering project could do. Not least the professional training of the people involved.
But, by and large people don't want to spend the money to make software as reliable as hardware, because it is fantastically expensive, and for most purposes you can just upgrade to the next release if it didn't work first time.
Even the DOD has been lured away from that most (well nearly) rigorous of programming languages ADA, to software written in languages that don't provide the same sort of guarantees, ultimately they must figure the tax payer gets a better deal now, but I'm betting the software in C++ is demonstrably buggier.
Have you ever met a good coder?
I don't think you are an experienced developer or someone who has ever met an experienced developer because any developer of experience will engineer software. They will produce a design (even a loose one) for release code, before that they will write proof of concept code and then use the proof of concept as the basis for their designs. A good coder will know their algorithms and be able to write tight, fast, well designed and documented code that has a low level of bugs. If you've ever met a coder like that then you will know that coding is just engineering, in the same way a car is engineering.
Code can also be a "spec"
Well structured, concise and clear code which is adequately commented (APIs and overall architecture) could also be be thought to be a specification of sort. One that actually reflects reality and where the conformance to reality is tested over millions of (Linux) machine all over the net all the time...
Like you can have tests for specifications (for example SDL can be formally tested), also code can be tested formally. The code could also be in some higher level language from which the actual implementation is generated automatically (I'm thinking more about protocol "specifications" than latest inventions for converting UML models to "working" code).
I'm not necessarily for literal programming, but there's a lot to be said from code that starts as a working prototype from which you can from the beginning generate usable documentation and which eventually will be refactored to a real implementation.
The documentation should be generated from (working) code, not the other way round. Code should be readable enough so that you read it like documentation/specification.
PS. A work mate of mine likes to promote "Design by Implementation" paradigm... :)
I agree. It often seems to be
I agree. It often seems to be overlooked or just ignored that code is a description by itself, and also that any program has a design, no matter whether someone thought of it beforehand, or it just ended up being the way it is by pure accident. So well-structured code doesn't need a lot of documentation to follow, it is self-documenting and the most precise description of the program you could ever get. The readability and overview may be greatly enhanced by using document-extracting tools on the code and its comments, but the main point is that the code is the primary description of itself - any other description may be educational, but approximate.
However, _requirements_ are still needed. The requirements will generally not be obvious when you read the code; they explain _why_ the code was written the way it is, and they help the developers by setting goals and constraints to be followed when creating the software.
'spec' == standard
There are different kinds of specifications. The kind that is discussed in the thread are standards, not specific hardware or software specs.
There is no x86 standard, only the Intel spec of their implementation. Then others decided they wanted to be compatible with that, and mostly followed it. This is totally different than e.g. HTML specs, where a browser who follows the specs blindly is near useless in practice, thanks to reality.
Linus Torvalds said:
And this is his main point I think.
I'd rather base my software o
I'd rather base my software on specs, then implement real-world "unexpected" behaviour on top of it as special cases.
Specs are abstractions which
Specs are abstractions which describe certain behaviour. Writing code while using the spec as sort of design document for the actual code implementation gives other results than writing code that fits best in its environment (e.g. hardware and rest of kernel), while still being compliant with the spec.
Imagine a spec which describes something with 5 layers and 10 classes. If you write code the same way, and someone points out that you only need two layers and 3 classes to do the same with much less code, in a nicer and more efficient way, wouldn't you agree? Or do you sputter and say "but according to the spec..."? Or in other words, what do you value more: the spec or reality?
If that doesn't break the spe
If that doesn't break the spec (as in interoperability) it might be considered. But other people who read our software will have to figure out what's going on in the code then.
And if a spec doesn't translate into good code, then it's not a good spec.
You can make a layer and say
You can make a layer and say in a comment somewhere that it implements layers a, b, c of spec X. The code doesn't have to be totally alienated from the spec, there's always a middle ground.
Some complex things may be best described very verbosely so that people can understand them. When that understandment is there, it can be trimmed down to the essential. So I don't think it's always true about the code translation, but in practice probably more often than not.
Imagine that almost no spec is good (technically: what they do is good, but they're poorly made), then you'll understand Linus' point of view to not translate specs into bad code. If you're lucky enough to work with a good spec you can get away with it, but the code shouldn't be judged by how well it resembles the spec, but on its own.
You don't have a clue what yo
You don't have a clue what you're doing then. A spec isn't supposed to translate into code at all. It is supposed to describe, in as few words as possible, with no implementation details whatsoever, with no subjective opinions whatsoever, what minimum requirements the project must meet when it is complete to be considered functional and fit for purpose.
A spec is not a recipe, nor a set of instructions. If I give you a spec to build a bridge, it's not going to read
"must use x beams, attach beam a to beam b like so..."
it's going to read
"must support x tonnes, must stand in windspeeds of x, must last x years"
And anything you can do to meet it is fine, but you better fuckin meet it.
Man... if I was as public a figure as Linus is, I'd be embarressed as hell to have said something so stupid.
Sure there are specs like that
...but there also are specs that ARE much lower level and do specify details. As a spec writer myself, I know that specs have their limitations. I owned a set of architecture specs for a DSP core, and I know those specs have holes.
My specs were much more detailed than "It must support X tonnes," to be sure. Nonetheless, another set of people wrote microarchitecture specs that went under my architecture specs that went even further--"Use such-and-such register file for this, use such-and-such memory for that, etc." complete with pipeline diagrams, etc.
And then there was the actual implementation. In the end, the implementation did deviate from both the architecture and microarchitecture specs. But the device did ultimately work.
So in a sense Linus is right. The spec gives you guideposts and bounds. It does not give you the implementation. And, it won't, by itself, give you interoperability. There's no substitute for testing against other implementations.
It shouldn't give you any imp
It shouldn't give you any implementation. When a spec deviates from the what and starts filling in the how, it stops being a spec and becomes a half-assed plan. And a half-assed plan is worse than none at all.
I don't deal with low level hardware, so my experiences are different from yours, but I've done a lot of requirements gathering and spec writing myself, and I consider the hardest part of the job to be refining out the implementation crap that every armchair quarterback wants to throw into the mix.
When you get right down to it, you're writing a spec because you don't have the time, skill or inclination to do something yourself, so you're telling someone you trust to make competent decision in how to go about doing what you need. When you go from telling them what you need to telling them how to do their jobs, you have to question why you're writing a spec instead of doing it yourself.
Im sorry, but you are wrong.
Im sorry, but you are wrong.
You are describing a requirement.
A requirement indicates what something must do.
A spec details how it must go about doing it.
Requirement:
Move 20 tons of rubble to the dump.
Spec:
1)One heavy duty tuck with shovel attachment
2)An operator
3)A map to & permission to use the dump
4)Fuel for the machinery.
If there is anything missing in the spec, it can (and should) be modified in accordance to real-world experience.
It's better (and in the software world, anywhere between 20x and 120x cheaper) to change the spec before implimenting, but you MUST change the spec to suit the real world at all times, otherwise you get pissed off hackers that start to ignore specs, or just consider them crap, because they've only ever encountered weak / outdated specs (no offence Linus).
requirement - spec - arch
What you could spec is what i called architecture :)
A requirement is the goal of the product. A specification should describe all the interaction and constraint with the outside world (the API and sometime some constraint of performance).
Putting some architecture constraint in spec add noise or induice over-engenering (bigger design, slower, more complexe, buggier,...).
The designer must have a maximum of liberty to optimise the design.
requirement - spec - arch - user manual
i could also add, if you write some spec after the design was maid i called it user manual :)
i'm very afraid !
so this is the reason why linux just happens to run ?
eehhh
It often doesn't work and have many bugs...
But specs are not the way to reduce bug number either. They are the way to tell what the software must do, how, and how it can interoperate.
Of course, specs are always false, and must be fixed all the times the code evolve. Exactly the same thing as the code: it is always false too! and must always be fixed...
ELF is a pretty good spec, no
ELF is a pretty good spec, not too broken. It's stood the test of time without any big problems.
OpenGL is a pretty good spec too.