Just a lead in: cloud computing will be vital for "small" businesses because "a man's got to know his limitations."
Now we might twitter, and for all that companies growing from startup to second stage, usually around 5+ employees, all at once notice a rumble in the fuselage.
We had service bureaus, used ADP for payroll, and edge into Quicken to run the books. Cloud computing, next big thing (and has been for years).
Monday, October 13, 2008
Tuesday, August 12, 2008
Disaster. Recovery. Invention.
In most of this last year's pieces here at the land of SANNAS (a fine team of wonks and also a fantastic snack with a good lager) the theme often centered upon increasing demands of distributed ephemeral data and the challenge of managing the process of custody and validation.
This article's being typed into a 1Gb stick; about 2Mb of that stick contains an encryption program; acceptable overhead IMHO for the promise of securing the Next Great Novel and Sudoku downloads, as well as the launch codes for the Acme ® ‘Lil Armageddon family of products, my sonnets to Paris Hilton and other juicy bits.
I do not, as they say, keep a tidy desk. My brain stays healthy by understanding my own LIFO filing system and an ability to understand the strata and the high probability parts of the piles wherein nestles the Airline Magazine or the clipping of a local paper's crank I wanted to riff upon at leisure. This represents an elegant strategy promoting mental health albeit with a risk of structural collapse of the entropy-friendly piles of the arcane lore.
Somewhere, someone must be working on a desktop computing metaphor that allows for significant standing loads. Bearing walls. Like that. At the very least, maybe something like "The Clapper" to find that 1Gb slice of memory...
So, here's the thing: data all over the place, connected and unconnected with the not so subtle growth of metadata to describe the context and provenance of information along with the burden of incremental data to manage the data and thereby the added processing cycles for data management itself. Extremely bright designers have delivered high value tool infrastructures, and I, for one, am not worthy of holding their pocket protectors in the area of difficult code and algorithm implementation, and generally customer focused implementations.
But in the realm of Disaster Recovery mechanisms and services, preemptive trumps reactive. Some scenarios of the mode of disaster, use cases, deliver an example.
Pandemic Flu, Weather, Earthquake, Toxic Spill, extended outages of power, water, other broken infrastructure should be the object of sandtable exercises, at a minimum, to game through what (might likely) work in these scenarios.
Rather makes removable media a bit of a problem during times of "saw fan, engaged same", not to mention getting to the unnetworked unautomated and unavailable mélange of annotated manuals and Post It notes which, don't you know, are the keys to the kingdom, whether one acknowledges that or not.
The adhocracy of portable data (iPhone, et.al.)seems to drive the industry towards some sort of nexus, wherein the overall practice and form of storage management and optimization will trend toward something that looks very much like Open Source toolkits and standards. For some this will be the defining disaster; however, other mature technology (e.g., MVS et seq) informs us that the core functionality and benefits of the "mature" technology do not by any means always disappear, but become the subject of new core businesses and invention. Ye Olde Virtual Machine has shown a tenacity in meeting the market need, albeit in quite new forms.
So, vis a vis Disaster Recovery, the pressure is on for shifts that make for highly interoperable and fungible networked storage resources (think Googleplex) with arbitrarily attached processing and storage tools. A lot of the "math" to predict the future comes from the good works of people like Gene Amdahl and Jim Gray (of Microsoft fame) in that a feasibility test can be accomplished with relative ease; with new cost factors and performance factors in hand, the maxim of "in the long run, all costs are variable" will again prove in with new invention. Of particular interest will be the results of open standards initiatives (akin to Web 3.0 posited mechanisms)where ontology will bloom like kudzu in Dixie.
And that, as the young lady informs us, is "hot".
This article's being typed into a 1Gb stick; about 2Mb of that stick contains an encryption program; acceptable overhead IMHO for the promise of securing the Next Great Novel and Sudoku downloads, as well as the launch codes for the Acme ® ‘Lil Armageddon family of products, my sonnets to Paris Hilton and other juicy bits.
I do not, as they say, keep a tidy desk. My brain stays healthy by understanding my own LIFO filing system and an ability to understand the strata and the high probability parts of the piles wherein nestles the Airline Magazine or the clipping of a local paper's crank I wanted to riff upon at leisure. This represents an elegant strategy promoting mental health albeit with a risk of structural collapse of the entropy-friendly piles of the arcane lore.
Somewhere, someone must be working on a desktop computing metaphor that allows for significant standing loads. Bearing walls. Like that. At the very least, maybe something like "The Clapper" to find that 1Gb slice of memory...
So, here's the thing: data all over the place, connected and unconnected with the not so subtle growth of metadata to describe the context and provenance of information along with the burden of incremental data to manage the data and thereby the added processing cycles for data management itself. Extremely bright designers have delivered high value tool infrastructures, and I, for one, am not worthy of holding their pocket protectors in the area of difficult code and algorithm implementation, and generally customer focused implementations.
But in the realm of Disaster Recovery mechanisms and services, preemptive trumps reactive. Some scenarios of the mode of disaster, use cases, deliver an example.
Pandemic Flu, Weather, Earthquake, Toxic Spill, extended outages of power, water, other broken infrastructure should be the object of sandtable exercises, at a minimum, to game through what (might likely) work in these scenarios.
Rather makes removable media a bit of a problem during times of "saw fan, engaged same", not to mention getting to the unnetworked unautomated and unavailable mélange of annotated manuals and Post It notes which, don't you know, are the keys to the kingdom, whether one acknowledges that or not.
The adhocracy of portable data (iPhone, et.al.)seems to drive the industry towards some sort of nexus, wherein the overall practice and form of storage management and optimization will trend toward something that looks very much like Open Source toolkits and standards. For some this will be the defining disaster; however, other mature technology (e.g., MVS et seq) informs us that the core functionality and benefits of the "mature" technology do not by any means always disappear, but become the subject of new core businesses and invention. Ye Olde Virtual Machine has shown a tenacity in meeting the market need, albeit in quite new forms.
So, vis a vis Disaster Recovery, the pressure is on for shifts that make for highly interoperable and fungible networked storage resources (think Googleplex) with arbitrarily attached processing and storage tools. A lot of the "math" to predict the future comes from the good works of people like Gene Amdahl and Jim Gray (of Microsoft fame) in that a feasibility test can be accomplished with relative ease; with new cost factors and performance factors in hand, the maxim of "in the long run, all costs are variable" will again prove in with new invention. Of particular interest will be the results of open standards initiatives (akin to Web 3.0 posited mechanisms)where ontology will bloom like kudzu in Dixie.
And that, as the young lady informs us, is "hot".
Thursday, July 17, 2008
Disk Payload Management
Transfer of data has an upper bound of the speed of light and a lower bound of amount of a budget, excluding strange action at a distance and physics not yet known. It's all fun and games until something divides by zero.
In a delightful teaser article, Neil J. Gunther's "The Guerrilla Manual" delivers a bolus of refreshing views on capacity planning and performance management with a cleansing amount of terse common sense.
In particular, he notes, "You never remove the bottleneck, you just shuffle the deck."
Network Effects and Blinkenlights
Back in the mid 1980s, at least one large financial institution allocated IT budgets using a simple ratio of numbers of customer accounts by type, with appropriate finagle factors. At least it was a model that, assuming a lot of linearity, had simplicity and apparent transparency going for it.
Of course, these were the times of data centers with big boxes, and the occasional minicomputer. The unit costs of processing, networks, and storage were significant vis a vis cycles or bits or bytes per dollar and cycles per watt.
Of course, also, the use cases for the technology moved rather slowly, with occasional punctuation with growing online inquiry from, say, customer service agents or the addition of Automatic Teller Machines to the CICS olio of the big iron code.
More gadgets and new approaches to programming by the end users (unclean!!!) resulted in rather surprising effects upon infrastructure through rampant flaming queries (what did he say?) and even complete suites of large scale computing systems dedicated to new types of models. In the case of financial services, one big dude jammed with APL for determination of fixed income dynamics. APL, for those who don't recall, was developed for passive aggressive savants who didn't want management looking into what they'd written. But, with letting the punishment fit the crime, APL rocked for matrix operations and was a darling of the early generation of quants, including those laugh a minute actuaries.
Somewhere, someplace, someone is hacking FPGAs to stick into the Beowulf cluster of X Boxes. I gotta feeling.
So where were we... Oh, so the point is that the common factor around these early instances of "end user" computing involved moderate and increasing network effects. Transactional data could be used as feeds to these user managed systems, and network effects with emphasis upon storage and I/O tuning became significant as a means of moving the bottleneck back to the cpu. Now pick another card.
The disk to disk discussion comprises several use cases, ranging from performance optimization (e.g, put the top 10 movies on the edge of the network) to business continuance to the meta issue of secure transfer and "lockup" of the data. Problem is, how does one deal with this mess which embraces Service Oriented Architectures and Halo dynamism?
Intelligent Payloads?
This problem of placing data and copies of data in "good enough" sites on the network seems encumbered by how these data are tagged in such a way as to inform the "system" itself on the history of the atomic piece of interest as it transits other systems and networks. Perhaps something that appends usage information to the information itself, rather like appending travel stickers to an old steamer trunk tracing its owner's tours of Alice Springs, Kenosha, and Banff.
And no, I'm not advocating still another inband system monitor... more MIBs than MIPS and all of that problem.
This could, I believe, be a fertile area for new types of automation that begin to apply optimization (satisficing, most likely, in the sense of "good enough" strategies, see Herbert Simon for more G2) thereby, maybe (he qualifies again!) to reduce the amount of time and money spent upon forensics and weird extraction of information needed to govern surprisingly fluid dynamic systems.
Zipf's Law (think top 10 lists, 80/20 rule, The Long Tail issues, etc.) and other power law behaviors will still apply to the end product of such analysis, but perhaps the informed payloads will ease the analysts' management of these turbulent parcels. (Some insights to the framing of the problem of getting top level insight into systems structures and how they express emergent behaviors can be found at the Santa Fe Institute and their many papers on "Small World" problems.)
So, the bounds on this problem of course reduces to time and money. That topic also is taken up by Gunther, with emphasis upon what some of my old gang at the Wall Street joint referred to as "the giggle test" for feasibility.
This is a brief piece about an intriguing problem where more insight can be gained from Operations Research methodologies than from Information Technology praxis per se.
It nets out to (sorry) not only if it is not measured, it isn't managed, but add to that the cautionary insight of "if it isn't modeled, it isn't managed."
In a delightful teaser article, Neil J. Gunther's "The Guerrilla Manual" delivers a bolus of refreshing views on capacity planning and performance management with a cleansing amount of terse common sense.
In particular, he notes, "You never remove the bottleneck, you just shuffle the deck."
Network Effects and Blinkenlights
Back in the mid 1980s, at least one large financial institution allocated IT budgets using a simple ratio of numbers of customer accounts by type, with appropriate finagle factors. At least it was a model that, assuming a lot of linearity, had simplicity and apparent transparency going for it.
Of course, these were the times of data centers with big boxes, and the occasional minicomputer. The unit costs of processing, networks, and storage were significant vis a vis cycles or bits or bytes per dollar and cycles per watt.
Of course, also, the use cases for the technology moved rather slowly, with occasional punctuation with growing online inquiry from, say, customer service agents or the addition of Automatic Teller Machines to the CICS olio of the big iron code.
More gadgets and new approaches to programming by the end users (unclean!!!) resulted in rather surprising effects upon infrastructure through rampant flaming queries (what did he say?) and even complete suites of large scale computing systems dedicated to new types of models. In the case of financial services, one big dude jammed with APL for determination of fixed income dynamics. APL, for those who don't recall, was developed for passive aggressive savants who didn't want management looking into what they'd written. But, with letting the punishment fit the crime, APL rocked for matrix operations and was a darling of the early generation of quants, including those laugh a minute actuaries.
Somewhere, someplace, someone is hacking FPGAs to stick into the Beowulf cluster of X Boxes. I gotta feeling.
So where were we... Oh, so the point is that the common factor around these early instances of "end user" computing involved moderate and increasing network effects. Transactional data could be used as feeds to these user managed systems, and network effects with emphasis upon storage and I/O tuning became significant as a means of moving the bottleneck back to the cpu. Now pick another card.
The disk to disk discussion comprises several use cases, ranging from performance optimization (e.g, put the top 10 movies on the edge of the network) to business continuance to the meta issue of secure transfer and "lockup" of the data. Problem is, how does one deal with this mess which embraces Service Oriented Architectures and Halo dynamism?
Intelligent Payloads?
This problem of placing data and copies of data in "good enough" sites on the network seems encumbered by how these data are tagged in such a way as to inform the "system" itself on the history of the atomic piece of interest as it transits other systems and networks. Perhaps something that appends usage information to the information itself, rather like appending travel stickers to an old steamer trunk tracing its owner's tours of Alice Springs, Kenosha, and Banff.
And no, I'm not advocating still another inband system monitor... more MIBs than MIPS and all of that problem.
This could, I believe, be a fertile area for new types of automation that begin to apply optimization (satisficing, most likely, in the sense of "good enough" strategies, see Herbert Simon for more G2) thereby, maybe (he qualifies again!) to reduce the amount of time and money spent upon forensics and weird extraction of information needed to govern surprisingly fluid dynamic systems.
Zipf's Law (think top 10 lists, 80/20 rule, The Long Tail issues, etc.) and other power law behaviors will still apply to the end product of such analysis, but perhaps the informed payloads will ease the analysts' management of these turbulent parcels. (Some insights to the framing of the problem of getting top level insight into systems structures and how they express emergent behaviors can be found at the Santa Fe Institute and their many papers on "Small World" problems.)
So, the bounds on this problem of course reduces to time and money. That topic also is taken up by Gunther, with emphasis upon what some of my old gang at the Wall Street joint referred to as "the giggle test" for feasibility.
This is a brief piece about an intriguing problem where more insight can be gained from Operations Research methodologies than from Information Technology praxis per se.
It nets out to (sorry) not only if it is not measured, it isn't managed, but add to that the cautionary insight of "if it isn't modeled, it isn't managed."
Friday, June 13, 2008
Trusted Infrastructure
Every day someone (or... something) with 218 dot et dot cet dot era out of China intently checks my TCP port connectivity. Relentless, time after time, again and again it seeks what I do not know, just that it signifies.
Cut us a break? Not likely.
Good old 218 (we're on a first octet basis) and his kin know that the herd, large and lacking vaccine, eases the pursuit of new zombies and kindred grift keep those who are far, far, more than "script kiddies" hard at work.
More than to get through spam filters (v149ra, 'frinstance or his sister of ill repute, C. Alice) or help to facilitate commercial dealings involving so often the sad demise of the spouse, uncle, client, former ambassador... tragedies all. I never knew how much wealth sloshes around as the result of corpses. Ambulance chasing seems quite profitable.
"Make Money Fast" now like a Madeline evokes the time and place of a 14.4 "golden age" before a simple "delete" command would be replaced with protocols that put a Level 4 Biohazard facility to shame (and this hinkey mess o code just for a desktop!)
Nothing to see here, move along!
Richard A. Clarke, of US National Security fame, describes in his new book "Your Government Has Failed You", the results of a security exercise with Department of Defense systems. White hats commenced to do that hoo doo that they do so well... and pretty much it was "game over" in a trice: penetration aptly describes what happened.
The guvment got Intrusion monitors installed. They lit up like Bay Ridge Brooklyn on Christmas Eve with all sorts of dubious packets from UnKn0wn U53r types. Clarke quotes managers telling him that until the hoods started setting off the intrusion alarms there had never been any putative bad guys. Somehow the security tech now attracted the bums, etc. Familiar story that I'm inclined to believe.
He also advocates a partitioned Internet, hardening part of the services for secure transactions and communications.
I've been there philosophically myself; and nonetheless am avidly in favor of an unrestrained Internet as well. No hobgoblins here.
The company I was with in 2001/2002 had a trusted client server implementation that did very serious authentication (non repudiation, yadda yadda rather like Moody Blues lyrics, those shalts and shants in them there specs ballyhoo optional). Pretty much good 2 go, add Series A and shake. Well, we had the hip but not the audience.
But the big deal was that our technology needed a pretty well suborned geek to break it (it wasn't open source, and in that I now trust it less but I'm just sayin'). It kept things nice and shiny and very private as an overlay to the Big I at large. Other Big Problem though comes from oceans of distributed data.
In a world: Queue Don LaFontaine
In a world where everything moves to an emergent ad hoc architecture of networked promiscuous storage services, where devices (picture frame, 1Gb wrist bracelet, virtualized server storage complexes, distributed p2p/p4p, DRM, TiVo, and even instrumentation motes reporting on movement of RFID tags and small area hydrology and nitrogen measures, etc.etc. containing massive amounts of data) often in surprisingly readable formats.
In a world... where consumers have started to put health records within the uberplexes of Microsoft and Google (getting the providers en masse to adopt it, different problem) and the concept of insurance moves from shared risk to, well, greed seems a good word.
In a world... where the laptops of one government are targeted and tossed (usage as in noir cant) for records of another government's dissidents and the bon mots of "gentlemen do not read each others mail" have no standing...
In a world where the retiring Secretary of Health and Human Services says "For the life of me, I cannot understand why the terrorists have not attacked our food supply because it is so easy to do." (NYTimes, December 2004)
In a world where a friend takes an extended business trip, returns to find that his kids have made five movies and posted them on YouTube from his Mac.
In a world where cameras come free with the printer, and a postage stamp of silicon tells me I'm on shot one of eighteen hundred.
Trusted Infrastructure Elements
So, here's where it goes, maybe. I have two things: things one and thing two.
Ubiquitous One Time Pads: All private, all the time.
Perhaps standing the security and privacy on end makes sense. Processing cycles and (good grief I actually typed core but stet!) core become pretty inexpensive, so running an algorithm to encode everything might not be a bad way to go. Heck, with some of the new word processing file formats alleging "open" protocols we seem to be half way there already; I can't open the attachments. We already see the unit costs of solid state "drives" precipitously dropping; headroom for small processing to encrypt onchip data could be a value added feature. The great thing is that these solid state devices are so small and hold so much data. The problem is that these solid state devices are so small and hold so much data.
Distributed Spread Spectrum Storage Service
Pseudorandom multi channel information paths, with a "big enough" local store to allow for unreliable networked service. The signal (information) hides in the noise. A form of this unreliable network appears as "The Byzantine Generals" problem, specifically Byzantine fault tolerance, wherein unreliable communications mechanisms become designed "around". Reason for this part of the puzzle is that there's so much data in so many places (cheap, dense, ubiquitous) that the use case for managed services around security and recovery needs to account for mobility, and placement of user data in many places. Key point though, is that a number of public and private researchers have been going at the reliability from unreliable infrastructure for a good while now (like, even ARPANET embodied this meme) and those wacky folks at Bittorrent (for example) have large amounts of distributed secure services in their beginning gene pool plus the understanding of how to reliably place data all over whilst maintaining service levels. If the Bit folks seem too edgy, then take a look at the Akamai technology model, and throw in a dose of Meraki Networks or perhaps Spain's FON.
Point is, seems that "the industry" has a pretty solid base of knowledge in sundry places that can be applied to the secure distributed data problem, and that the notional risks to private data help to monetize the innovation's value. The architects/designers Ray and Charles Eames explored the concept of "kit of parts" for building; I think that much of that "kit of parts" exists in the public domain and will be amended with patent expiries over the next several years.
Your Mileage May Vary
That vision above, if it merits that moniker, ignores some Big Problems like distributed key management for starters, twenty volumes of USENET rants about Pretty Good Privacy (PGP), in band versus out of band control, and a whole lot of other things. I believe the joke ends with the physicist saying "we've solved the problem for a sphere." Managing distributed data "in a world" of ubiquitous storage seems the next grand challenge.
Good hunting.
Cut us a break? Not likely.
Good old 218 (we're on a first octet basis) and his kin know that the herd, large and lacking vaccine, eases the pursuit of new zombies and kindred grift keep those who are far, far, more than "script kiddies" hard at work.
More than to get through spam filters (v149ra, 'frinstance or his sister of ill repute, C. Alice) or help to facilitate commercial dealings involving so often the sad demise of the spouse, uncle, client, former ambassador... tragedies all. I never knew how much wealth sloshes around as the result of corpses. Ambulance chasing seems quite profitable.
"Make Money Fast" now like a Madeline evokes the time and place of a 14.4 "golden age" before a simple "delete" command would be replaced with protocols that put a Level 4 Biohazard facility to shame (and this hinkey mess o code just for a desktop!)
Nothing to see here, move along!
Richard A. Clarke, of US National Security fame, describes in his new book "Your Government Has Failed You", the results of a security exercise with Department of Defense systems. White hats commenced to do that hoo doo that they do so well... and pretty much it was "game over" in a trice: penetration aptly describes what happened.
The guvment got Intrusion monitors installed. They lit up like Bay Ridge Brooklyn on Christmas Eve with all sorts of dubious packets from UnKn0wn U53r types. Clarke quotes managers telling him that until the hoods started setting off the intrusion alarms there had never been any putative bad guys. Somehow the security tech now attracted the bums, etc. Familiar story that I'm inclined to believe.
He also advocates a partitioned Internet, hardening part of the services for secure transactions and communications.
I've been there philosophically myself; and nonetheless am avidly in favor of an unrestrained Internet as well. No hobgoblins here.
The company I was with in 2001/2002 had a trusted client server implementation that did very serious authentication (non repudiation, yadda yadda rather like Moody Blues lyrics, those shalts and shants in them there specs ballyhoo optional). Pretty much good 2 go, add Series A and shake. Well, we had the hip but not the audience.
But the big deal was that our technology needed a pretty well suborned geek to break it (it wasn't open source, and in that I now trust it less but I'm just sayin'). It kept things nice and shiny and very private as an overlay to the Big I at large. Other Big Problem though comes from oceans of distributed data.
In a world: Queue Don LaFontaine
In a world where everything moves to an emergent ad hoc architecture of networked promiscuous storage services, where devices (picture frame, 1Gb wrist bracelet, virtualized server storage complexes, distributed p2p/p4p, DRM, TiVo, and even instrumentation motes reporting on movement of RFID tags and small area hydrology and nitrogen measures, etc.etc. containing massive amounts of data) often in surprisingly readable formats.
In a world... where consumers have started to put health records within the uberplexes of Microsoft and Google (getting the providers en masse to adopt it, different problem) and the concept of insurance moves from shared risk to, well, greed seems a good word.
In a world... where the laptops of one government are targeted and tossed (usage as in noir cant) for records of another government's dissidents and the bon mots of "gentlemen do not read each others mail" have no standing...
In a world where the retiring Secretary of Health and Human Services says "For the life of me, I cannot understand why the terrorists have not attacked our food supply because it is so easy to do." (NYTimes, December 2004)
In a world where a friend takes an extended business trip, returns to find that his kids have made five movies and posted them on YouTube from his Mac.
In a world where cameras come free with the printer, and a postage stamp of silicon tells me I'm on shot one of eighteen hundred.
Trusted Infrastructure Elements
So, here's where it goes, maybe. I have two things: things one and thing two.
Ubiquitous One Time Pads: All private, all the time.
Perhaps standing the security and privacy on end makes sense. Processing cycles and (good grief I actually typed core but stet!) core become pretty inexpensive, so running an algorithm to encode everything might not be a bad way to go. Heck, with some of the new word processing file formats alleging "open" protocols we seem to be half way there already; I can't open the attachments. We already see the unit costs of solid state "drives" precipitously dropping; headroom for small processing to encrypt onchip data could be a value added feature. The great thing is that these solid state devices are so small and hold so much data. The problem is that these solid state devices are so small and hold so much data.
Distributed Spread Spectrum Storage Service
Pseudorandom multi channel information paths, with a "big enough" local store to allow for unreliable networked service. The signal (information) hides in the noise. A form of this unreliable network appears as "The Byzantine Generals" problem, specifically Byzantine fault tolerance, wherein unreliable communications mechanisms become designed "around". Reason for this part of the puzzle is that there's so much data in so many places (cheap, dense, ubiquitous) that the use case for managed services around security and recovery needs to account for mobility, and placement of user data in many places. Key point though, is that a number of public and private researchers have been going at the reliability from unreliable infrastructure for a good while now (like, even ARPANET embodied this meme) and those wacky folks at Bittorrent (for example) have large amounts of distributed secure services in their beginning gene pool plus the understanding of how to reliably place data all over whilst maintaining service levels. If the Bit folks seem too edgy, then take a look at the Akamai technology model, and throw in a dose of Meraki Networks or perhaps Spain's FON.
Point is, seems that "the industry" has a pretty solid base of knowledge in sundry places that can be applied to the secure distributed data problem, and that the notional risks to private data help to monetize the innovation's value. The architects/designers Ray and Charles Eames explored the concept of "kit of parts" for building; I think that much of that "kit of parts" exists in the public domain and will be amended with patent expiries over the next several years.
Your Mileage May Vary
That vision above, if it merits that moniker, ignores some Big Problems like distributed key management for starters, twenty volumes of USENET rants about Pretty Good Privacy (PGP), in band versus out of band control, and a whole lot of other things. I believe the joke ends with the physicist saying "we've solved the problem for a sphere." Managing distributed data "in a world" of ubiquitous storage seems the next grand challenge.
Good hunting.
Wednesday, May 14, 2008
Compliance CENTREX
What went around comes around.
Compliance, to pass an examination or be prepared for a contingency, has foundation from commercial good practice and regulatory (including legal) conformance.
Higher level compliance concerns evoke the wisdom of first do no harm, eat green leafy vegetables, and get moderate exercise. Business continuance, or continuance of the entity or ones job, ripens with common sense. The broader topics include fiduciary stewardship, management of proprietary internal information, and of client, patient, supplier, et.al. privacy, and, fundamentally, the determination of trust.
In forming the deeply connected systems involving the customers, the organizations, and the suppliers the challenge of determining trust provably is, well, not easy.
In a recent national survey, respondents uniformly agreed that this validation of a system, the compliance issues in managing the archiving and retrieval (or demise) of data worked, by degree, well in larger organizations but was woefully lacking in many others.
Be advised that this national survey involved three people I know.
But they were pretty far away from each other.
A couple of common themes though, on the broad area of what's up with that compliance thing?
Small Organizations Getting Crushed
Many smaller organizations encounter compliance, in the broad sense, only after something has gone quite wrong. The data systems come under the management of one "s/he who knows" and typically that s/he who knows avers that all's under control, nothing to see, move along. Quite often, that's a correct assessment but the Black Swan of Something Wicked will assuredly appear like the first tulip of spring.
The many conceivable test cases and demonstrations available to exercise a procedure or software implementation become quickly incomplete; the "many eyeballs" of the Linux community or MSDN or the SAP experts or Cisco herders or System Z mavens or Wii Wunderkind and the rest may already have the fix in hand, but the bolus of healing code remains unknown to the many.
So staying current with what's on with the system in the technology sense underpins meaningful compliance.
The management of even a small system, when networked, overwhelms or at least distracts from the business at hand, the getting the work done part of work. Script kiddies, malware, and zombie makers test test test the borders and their numbers still grow. Firewalls capture the IP of pinging drones with the tireless cold focus of The Terminator.
But enough of the light side.
Somewhere there's a law waiting to bite you.
In addition to the issues of compliance with securing systems from perpetrators and miscreants, somewhere one can find comfort in some legislation from someone who should, but does not, know "better" that will increase costs for compliance if not explicitly be purely intractable by the placement of the liability of non compliance with an impossible dream.
Those public policy decisions might include draconian warnings for browsing a Goya Maja or a 1970s Sears Catalog, and more seriously issues of record retention and access for, among other things, financial and health transactional data, and emails (which increasingly go beyond text and have average object sizes growing like Kudzu). I wonder if Wall Street still drives those tapes around in the vans, and whether aging tape drives live on to read archaic media. This leads to still more policy decisions, internal to an organization and implementations driven by external policy makers.
For the smaller organization, staying on top of the goat rodeo that entails version management of desktop OS and warring tribes of antiwhatever software chewing through silicon to the joy of chipkateers and the anguish of people just trying to get along with recreational and the oh so serious land of business computing, now that goat rodeo builds character in and of itself.
To overlay "unfunded mandates" on the already maxtasked elves of IT, well that's just Bondo on the cake.
Hence: Compliance Centrex
In the dark ages of the 1970s and 1980s, telephone companies in more urban climes implemented a product called Centrex (Centralized Exchange) service.
The mainframe (Central Office) provided features from a bunker managed by truly serious professionals empowered to code the forwarding of phone calls from one line to another, implement restrictions on calling privileges to certain telephone lines (local only, for example) and wield other powers on the users, who enjoyed very reliable consistent service. Each line got its own phone bill. Later generations of the Centrex miracle provided limited administration by the customer.
Point: a small team of skilled people can provide a lot of oomph across a customer base. 37signals with their millions of subscribers being one example already covered here.
Somewhere I hope there's a graduate student looking at the marginal utility of willingness to pay with notional conjoint analysis studies.
More fun than Facebook!
Now, the code to be cracked appeals most to small and medium scale enterprises; bundling policy based compliance features as a software overlay or a true "tin can with widgets + code inside" offers some hope. Think of an XML (etc.) publishing service informing the installed base of gadgetry on the latest rules for archiving, say, emails. Recognize also that the "big" end of the market also may actively seek compliance services: Collaborative Software Initiative's reference implementation for custodial information is but one example.
Towards the Compliance Appliance?
Here in the US, a couple of firms now warrant identity protection with a $1,000,000 guarantee($400 Loonies thereabouts). Will we see participation of new risk instruments, insurance akin to errors and omissions, appearing as part of new "tough" compliance appliances?
Compliance as a service has, I believe, legs in the same sense that an antivirus automatic update does. Some "professional compliance" process exists for food companies: third parties provide certification of the supply chain end of the food pipe to ensure that custodial information, security and hygiene of premises, etc. conform to regulatory standards. Beyond "audit" the service economy for compliance is likely quite amenable to a service provided for fee relationship, increasing service revenues for the storage company (or the applications company) along with driving new purpose into the consigliore side of the bidnez.
Continuous monitoring already available for "tested hacker safe" affirms the underlying business model.
The management of the infrastructure, especially for the smaller companies, takes too many cycles and depends upon people who rarely encounter a Black Swan in their Sarbox, eh?
Teams matter; effective new ones will cross borders of all sorts.
Compliance, to pass an examination or be prepared for a contingency, has foundation from commercial good practice and regulatory (including legal) conformance.
Higher level compliance concerns evoke the wisdom of first do no harm, eat green leafy vegetables, and get moderate exercise. Business continuance, or continuance of the entity or ones job, ripens with common sense. The broader topics include fiduciary stewardship, management of proprietary internal information, and of client, patient, supplier, et.al. privacy, and, fundamentally, the determination of trust.
In forming the deeply connected systems involving the customers, the organizations, and the suppliers the challenge of determining trust provably is, well, not easy.
In a recent national survey, respondents uniformly agreed that this validation of a system, the compliance issues in managing the archiving and retrieval (or demise) of data worked, by degree, well in larger organizations but was woefully lacking in many others.
Be advised that this national survey involved three people I know.
But they were pretty far away from each other.
A couple of common themes though, on the broad area of what's up with that compliance thing?
Small Organizations Getting Crushed
Many smaller organizations encounter compliance, in the broad sense, only after something has gone quite wrong. The data systems come under the management of one "s/he who knows" and typically that s/he who knows avers that all's under control, nothing to see, move along. Quite often, that's a correct assessment but the Black Swan of Something Wicked will assuredly appear like the first tulip of spring.
The many conceivable test cases and demonstrations available to exercise a procedure or software implementation become quickly incomplete; the "many eyeballs" of the Linux community or MSDN or the SAP experts or Cisco herders or System Z mavens or Wii Wunderkind and the rest may already have the fix in hand, but the bolus of healing code remains unknown to the many.
So staying current with what's on with the system in the technology sense underpins meaningful compliance.
The management of even a small system, when networked, overwhelms or at least distracts from the business at hand, the getting the work done part of work. Script kiddies, malware, and zombie makers test test test the borders and their numbers still grow. Firewalls capture the IP of pinging drones with the tireless cold focus of The Terminator.
But enough of the light side.
Somewhere there's a law waiting to bite you.
In addition to the issues of compliance with securing systems from perpetrators and miscreants, somewhere one can find comfort in some legislation from someone who should, but does not, know "better" that will increase costs for compliance if not explicitly be purely intractable by the placement of the liability of non compliance with an impossible dream.
Those public policy decisions might include draconian warnings for browsing a Goya Maja or a 1970s Sears Catalog, and more seriously issues of record retention and access for, among other things, financial and health transactional data, and emails (which increasingly go beyond text and have average object sizes growing like Kudzu). I wonder if Wall Street still drives those tapes around in the vans, and whether aging tape drives live on to read archaic media. This leads to still more policy decisions, internal to an organization and implementations driven by external policy makers.
For the smaller organization, staying on top of the goat rodeo that entails version management of desktop OS and warring tribes of antiwhatever software chewing through silicon to the joy of chipkateers and the anguish of people just trying to get along with recreational and the oh so serious land of business computing, now that goat rodeo builds character in and of itself.
To overlay "unfunded mandates" on the already maxtasked elves of IT, well that's just Bondo on the cake.
Hence: Compliance Centrex
In the dark ages of the 1970s and 1980s, telephone companies in more urban climes implemented a product called Centrex (Centralized Exchange) service.
The mainframe (Central Office) provided features from a bunker managed by truly serious professionals empowered to code the forwarding of phone calls from one line to another, implement restrictions on calling privileges to certain telephone lines (local only, for example) and wield other powers on the users, who enjoyed very reliable consistent service. Each line got its own phone bill. Later generations of the Centrex miracle provided limited administration by the customer.
Point: a small team of skilled people can provide a lot of oomph across a customer base. 37signals with their millions of subscribers being one example already covered here.
Somewhere I hope there's a graduate student looking at the marginal utility of willingness to pay with notional conjoint analysis studies.
More fun than Facebook!
Now, the code to be cracked appeals most to small and medium scale enterprises; bundling policy based compliance features as a software overlay or a true "tin can with widgets + code inside" offers some hope. Think of an XML (etc.) publishing service informing the installed base of gadgetry on the latest rules for archiving, say, emails. Recognize also that the "big" end of the market also may actively seek compliance services: Collaborative Software Initiative's reference implementation for custodial information is but one example.
Towards the Compliance Appliance?
Here in the US, a couple of firms now warrant identity protection with a $1,000,000 guarantee($400 Loonies thereabouts). Will we see participation of new risk instruments, insurance akin to errors and omissions, appearing as part of new "tough" compliance appliances?
Compliance as a service has, I believe, legs in the same sense that an antivirus automatic update does. Some "professional compliance" process exists for food companies: third parties provide certification of the supply chain end of the food pipe to ensure that custodial information, security and hygiene of premises, etc. conform to regulatory standards. Beyond "audit" the service economy for compliance is likely quite amenable to a service provided for fee relationship, increasing service revenues for the storage company (or the applications company) along with driving new purpose into the consigliore side of the bidnez.
Continuous monitoring already available for "tested hacker safe" affirms the underlying business model.
The management of the infrastructure, especially for the smaller companies, takes too many cycles and depends upon people who rarely encounter a Black Swan in their Sarbox, eh?
Teams matter; effective new ones will cross borders of all sorts.
Monday, April 14, 2008
Ephemeral Hardware
Someone, somewhere, once quipped that hardware is out of date software. One can just about take that at face value; Wall Street quants run engines out of FPGA arrays to rapidly calculate banyan fibonacci mojo ratios, let alone the insights given to March Madness here in the United States coming, no doubt, from hijacked cloud computing resources.
Meanwhile, the "not all gigabytes are created equal" applies to this industry where unit costs of storage hardware drop precipitously (and mostly predictably) whilst total storage management costs increase....but at typically lower unit costs.
Then the state diagram of "where does the market find equilibrium" sloshes in another direction with emergent nano densities (we visited that topic in November or December 2007 in here) which sloshes further with announcements of IBM Racetrack topologies (updating previous announcements from neoBUNCH forces on the development of first gen nano which was expected to ship late this year) for chips with billions of atomistic nanowires and even higher densities.
It's interesting to note that Buckminster Fuller, for whom the nanostructures of carbon called "Buckyballs" due to their resemblance to his Geodesic Dome (we've got the one he built and lived in when he taught at the local University here around 37.5W and 89N) also coined the word "ephimeralization".
Do more and more with less and less until you're doing everything with nothing basically sums up that principle; however, what is added to that koan? Haiku? whatever is information.
Seems that, to me, information is what holds that airy ship aloft, information represented as software and rules. So, the perfect storm (strike that, cliché) the fuzzball of storage becomes a witches brew of rapidly increasing "consumer driven" data (oodles of video, oodles of endpoints). And boy howdy, don't them metas get busy?
Problems of managing versions (dedupe, mirrors backups, recoveries), placement of data to manage service levels (cache, multiple copies, P2P or P4P) and (jargon alert) semantic web wherein constructs of tags of tags and winding hyperlinks reflecting states and relationships, not just for catching the great Diet Coke and Mentos research but to get the skinny on what the latest method is to build something, cure something, learn something, share something, price something, buy something, sell something.....
Or use the stored images to catch a thief (or worse) or provide compliance (not just for the SEC; as part of researching video applications for education, a fine gentleman named Mike Fink advised that some hospitals and emergency rooms are adopting video as anticipatory evidence when conducting particularly parlous procedures. (Perspicacity!)
The technology and methods approach not only Kurtzweil's "singularity" of intelligent machines, but the holodeck (hold on, Chief!) appears in the form of a walk about simulation of the hours before Pompeii's demise. The user of the system can move through that simulation by walking. (Heard that one on late night radio just before a discussion of alien abduction but I'm all about eclectic.) This, plus Wii systems have seen rapid adoption in physical therapy....
Consumers all over the joint.
Explicitly: this does not (yet) forecast the immediate demise of spinning media or tape, but the pricing pressures are and will continue to approach "unit costs of zero", with improving wide area connectivity which, while not allowing (for now) ignoring special relativity, multiple managed instances of information and stateful knowledge will appear more and more as abstractions within a deeply interconnected network. Seems not unreasonable to expect some form of bid/ask mechanisms to appear for dynamic allocation of, particularly, entertainment or thematically oriented information linking secure objects, generating additional copies, cleaning out the old data, and optimizing data (and processing) placement based upon dynamic needs.
Given that software itself decays, through maintenance in particular as cited in COCOMO models, etc., the management of storage more becomes the art of software as an increment to the art of understanding the more physical mechanisms of striping, RAID, densities, and mirrors. And the dark force of entropy, lo, will drive more preemptive testing and quality assurance (said the winner of sundry service packs).
And with the flat out impossibility of exhaustively testing every combination of lines of instructions (NP completeness means never having to say you're done) more nuanced and practical methods have emerged to do statistical measures akin to Design Of Experiment methods; however, still better hang on to that last working version.... Given that pretty much every buzzword required in a tech oriented article has been used here (oops, yeah, and Social Networking too!) we might as well toss in quantum computing.
And part of the quantum and string theory models is the many universes theory, which comforts me because it suggests, if true, I might finally be one of the smartest guys in the room, albeit a Cub Reporter.
Happy Spring!
Meanwhile, the "not all gigabytes are created equal" applies to this industry where unit costs of storage hardware drop precipitously (and mostly predictably) whilst total storage management costs increase....but at typically lower unit costs.
Then the state diagram of "where does the market find equilibrium" sloshes in another direction with emergent nano densities (we visited that topic in November or December 2007 in here) which sloshes further with announcements of IBM Racetrack topologies (updating previous announcements from neoBUNCH forces on the development of first gen nano which was expected to ship late this year) for chips with billions of atomistic nanowires and even higher densities.
It's interesting to note that Buckminster Fuller, for whom the nanostructures of carbon called "Buckyballs" due to their resemblance to his Geodesic Dome (we've got the one he built and lived in when he taught at the local University here around 37.5W and 89N) also coined the word "ephimeralization".
Do more and more with less and less until you're doing everything with nothing basically sums up that principle; however, what is added to that koan? Haiku? whatever is information.
Seems that, to me, information is what holds that airy ship aloft, information represented as software and rules. So, the perfect storm (strike that, cliché) the fuzzball of storage becomes a witches brew of rapidly increasing "consumer driven" data (oodles of video, oodles of endpoints). And boy howdy, don't them metas get busy?
Problems of managing versions (dedupe, mirrors backups, recoveries), placement of data to manage service levels (cache, multiple copies, P2P or P4P) and (jargon alert) semantic web wherein constructs of tags of tags and winding hyperlinks reflecting states and relationships, not just for catching the great Diet Coke and Mentos research but to get the skinny on what the latest method is to build something, cure something, learn something, share something, price something, buy something, sell something.....
Or use the stored images to catch a thief (or worse) or provide compliance (not just for the SEC; as part of researching video applications for education, a fine gentleman named Mike Fink advised that some hospitals and emergency rooms are adopting video as anticipatory evidence when conducting particularly parlous procedures. (Perspicacity!)
The technology and methods approach not only Kurtzweil's "singularity" of intelligent machines, but the holodeck (hold on, Chief!) appears in the form of a walk about simulation of the hours before Pompeii's demise. The user of the system can move through that simulation by walking. (Heard that one on late night radio just before a discussion of alien abduction but I'm all about eclectic.) This, plus Wii systems have seen rapid adoption in physical therapy....
Consumers all over the joint.
Explicitly: this does not (yet) forecast the immediate demise of spinning media or tape, but the pricing pressures are and will continue to approach "unit costs of zero", with improving wide area connectivity which, while not allowing (for now) ignoring special relativity, multiple managed instances of information and stateful knowledge will appear more and more as abstractions within a deeply interconnected network. Seems not unreasonable to expect some form of bid/ask mechanisms to appear for dynamic allocation of, particularly, entertainment or thematically oriented information linking secure objects, generating additional copies, cleaning out the old data, and optimizing data (and processing) placement based upon dynamic needs.
Given that software itself decays, through maintenance in particular as cited in COCOMO models, etc., the management of storage more becomes the art of software as an increment to the art of understanding the more physical mechanisms of striping, RAID, densities, and mirrors. And the dark force of entropy, lo, will drive more preemptive testing and quality assurance (said the winner of sundry service packs).
And with the flat out impossibility of exhaustively testing every combination of lines of instructions (NP completeness means never having to say you're done) more nuanced and practical methods have emerged to do statistical measures akin to Design Of Experiment methods; however, still better hang on to that last working version.... Given that pretty much every buzzword required in a tech oriented article has been used here (oops, yeah, and Social Networking too!) we might as well toss in quantum computing.
And part of the quantum and string theory models is the many universes theory, which comforts me because it suggests, if true, I might finally be one of the smartest guys in the room, albeit a Cub Reporter.
Happy Spring!
Wednesday, March 12, 2008
What: availability. A how: Continuous Data Protection
Took a look at the published "inventory" of information on Google to give myself some orientation to the development timeline of what folks have been saying about availability (specifically"High Availability") and "Continuous Data Protection" to see when people started turning ideas into products.
The HA issues zipped right along from around 1985 or so (this is a survey, not a census, dear reader) with articulated specifications, formation of managed service offerings, products, etc. zipping right along to our current worlds.Continuous Data Protection, and by that, *that* particular search term shows up circa 1991 as prelim to disk mirroring products appearing later that decade.
The pre Sysplex days (and more people were working on the distributed problem than IBM) rested upon dark fiber, to me, reflecting the some people longing for dial tone at 40 pfennings a minute. SMDS, SONET offerings hadn't yet shown up, but the results were pretty convincing among some (rumored) blue sparks and flame that having trusted data in at least two places at once with a prayer (improvement) in recovering ones systems from the distributed data, well.... very good thing.
I'd argue, however, that the Continuous Data Protection model is the converged solution for how to answer the question of applications availability; the economics of (planned) redundancy favor that information distribution. Kindred concerns of custodial, compliance, and reliable connectivity, while significant, do invite innovations in putting the data objects. Market momentum for how to build higher availability into applications comes from known good libraries of "how to do this".
The DeDupe market space, as well, offers cost relief from the ability to recycle and realize more efficiencies in net storage capacities. The cautionary tale here comes from distributed computing, wherein some applications resemble Yorkie Terriers. Very very good at playing "this, all of this, is mine!" to the tune of "Big Boss Man" resulting with a conundrum of which manager manages the manager managers and a stack of dueling control systems oh heck lets put another piece of monitoring software in there that ought to hold 'em....
Which in turn brings back memories of notoriously brittle High Availability systems from the 90s, wherein the prudent management discipline was to establish that it was at last working and hanging up a sign that said "Was working fine when I left it. You broke it."
Some local use cases (involving moderate storage requirements and a thin network infrastructure) indicate that Continuous is the way to go (assuming that the data "containers or objects" allow for incremental updates). Saves network, and keeps one closer to the point in time when the fan will make a rude noise. Seriously looking at the peer to peer model has some wonderful attributes of survivability, and redundancy (boy, you can say that again) also with the potential for borrowing resources across networks.So in no way is it a motherhood issue as to how.
Barbie: Math is hard. She caught H E Double Hockeysticks for that but that's a fact.
Meanwhile, the what is of the motherhood issue (viz, a requirement to keep things going). But that how (one's chosen implementation). Hoo Wee! That how's a poser. But to me there's something in the thought that "this system swaps things about all of the time and keeps running with a provable audit trail of service levels" as more comforting than "it's in Lou's truck". One can always, as it were, burn a disk. Demonstrating recovery during system operation as a normal course of business.... cool.
The HA issues zipped right along from around 1985 or so (this is a survey, not a census, dear reader) with articulated specifications, formation of managed service offerings, products, etc. zipping right along to our current worlds.Continuous Data Protection, and by that, *that* particular search term shows up circa 1991 as prelim to disk mirroring products appearing later that decade.
The pre Sysplex days (and more people were working on the distributed problem than IBM) rested upon dark fiber, to me, reflecting the some people longing for dial tone at 40 pfennings a minute. SMDS, SONET offerings hadn't yet shown up, but the results were pretty convincing among some (rumored) blue sparks and flame that having trusted data in at least two places at once with a prayer (improvement) in recovering ones systems from the distributed data, well.... very good thing.
I'd argue, however, that the Continuous Data Protection model is the converged solution for how to answer the question of applications availability; the economics of (planned) redundancy favor that information distribution. Kindred concerns of custodial, compliance, and reliable connectivity, while significant, do invite innovations in putting the data objects. Market momentum for how to build higher availability into applications comes from known good libraries of "how to do this".
The DeDupe market space, as well, offers cost relief from the ability to recycle and realize more efficiencies in net storage capacities. The cautionary tale here comes from distributed computing, wherein some applications resemble Yorkie Terriers. Very very good at playing "this, all of this, is mine!" to the tune of "Big Boss Man" resulting with a conundrum of which manager manages the manager managers and a stack of dueling control systems oh heck lets put another piece of monitoring software in there that ought to hold 'em....
Which in turn brings back memories of notoriously brittle High Availability systems from the 90s, wherein the prudent management discipline was to establish that it was at last working and hanging up a sign that said "Was working fine when I left it. You broke it."
Some local use cases (involving moderate storage requirements and a thin network infrastructure) indicate that Continuous is the way to go (assuming that the data "containers or objects" allow for incremental updates). Saves network, and keeps one closer to the point in time when the fan will make a rude noise. Seriously looking at the peer to peer model has some wonderful attributes of survivability, and redundancy (boy, you can say that again) also with the potential for borrowing resources across networks.So in no way is it a motherhood issue as to how.
Barbie: Math is hard. She caught H E Double Hockeysticks for that but that's a fact.
Meanwhile, the what is of the motherhood issue (viz, a requirement to keep things going). But that how (one's chosen implementation). Hoo Wee! That how's a poser. But to me there's something in the thought that "this system swaps things about all of the time and keeps running with a provable audit trail of service levels" as more comforting than "it's in Lou's truck". One can always, as it were, burn a disk. Demonstrating recovery during system operation as a normal course of business.... cool.
"Say watt again. SAY WATT AGAIN"
Joules, old boy.
The conversations around the topic of "Green Computing" have focussed very much upon the management of heat loads in the metaphor of power. The technology itself heads toward a lower unit consumption of power production of heat due to smaller geometry as balanced against higher densities.
Once upon a time, R. Buckminster Fuller looked at the Spaceship Earth problems of electrical distribution and posited that the total capacities of power could on a global basis (more or less) be reduced (ceteris paribus) if the power grids of the whole world were deeply interconnected.
Sunny side of the planet, dark side of the planet, lighting from the sun, lighting from electricity, factories on, factories off. With some nit picking regarding transmission losses, etc. the proposition yields an interesting gedanken experiment vis a vis networks of computers and networks of storage. With some form of daily (whatever) variations, moving workloads, comprised of payloads of data and eventually application pieces, could let one reduce a given site's "peak capacity" through clever movement of processing.
Yes, latency. Yes, transmission capacities. Yes, etc.
But forms of this more agile distributive model come to us from, gasp, BitTorrent and other computing structures. For anyone who lives with real technology, the working model is that the solution will not be perfect (obsolete as it gets plugged in, in fact) but that the solution should be measurably "better".
We're living in a material world. You go girl.
Part of the Green computing ecosystem has been and will be the interaction of design for life cycles from a Cradle To Cradle point of view, increasing pressure on key materials in the computing industry (rare earths, gold), and improving recognition of the value of these materials in the cycle of creation, use, deconstruction, and re use. Fuller looked in particular at copper cycling through the system; the recycling of component materials in computing, however, has only recently become a design requirement. (LeRoy Budnik noted the "drive that will not die" in a recent post here.)
But the materials will be better "designed" to enable reuse in part because of "goodness" but principally because it makes sound organizational sense from the view of economics.
That the trend of ephemeralization (the less is more than more koan) cuts into this olio will be saved for another time.
When I have the erg.
With apologies.
The conversations around the topic of "Green Computing" have focussed very much upon the management of heat loads in the metaphor of power. The technology itself heads toward a lower unit consumption of power production of heat due to smaller geometry as balanced against higher densities.
Once upon a time, R. Buckminster Fuller looked at the Spaceship Earth problems of electrical distribution and posited that the total capacities of power could on a global basis (more or less) be reduced (ceteris paribus) if the power grids of the whole world were deeply interconnected.
Sunny side of the planet, dark side of the planet, lighting from the sun, lighting from electricity, factories on, factories off. With some nit picking regarding transmission losses, etc. the proposition yields an interesting gedanken experiment vis a vis networks of computers and networks of storage. With some form of daily (whatever) variations, moving workloads, comprised of payloads of data and eventually application pieces, could let one reduce a given site's "peak capacity" through clever movement of processing.
Yes, latency. Yes, transmission capacities. Yes, etc.
But forms of this more agile distributive model come to us from, gasp, BitTorrent and other computing structures. For anyone who lives with real technology, the working model is that the solution will not be perfect (obsolete as it gets plugged in, in fact) but that the solution should be measurably "better".
We're living in a material world. You go girl.
" Metals recirculate on a sum-total-of-all-metals-average every 22 1/2 years....
I was able to arrive at that figure of a 22 1/2-year metals recirculating cycle in 1936. I was working for Phelps Dodge Co., which had asked me to give them some prognostications about the uses of copper in the future of world industry."R. Buckminster Fuller, Critical Path
Part of the Green computing ecosystem has been and will be the interaction of design for life cycles from a Cradle To Cradle point of view, increasing pressure on key materials in the computing industry (rare earths, gold), and improving recognition of the value of these materials in the cycle of creation, use, deconstruction, and re use. Fuller looked in particular at copper cycling through the system; the recycling of component materials in computing, however, has only recently become a design requirement. (LeRoy Budnik noted the "drive that will not die" in a recent post here.)
But the materials will be better "designed" to enable reuse in part because of "goodness" but principally because it makes sound organizational sense from the view of economics.
That the trend of ephemeralization (the less is more than more koan) cuts into this olio will be saved for another time.
When I have the erg.
With apologies.
Tuesday, February 26, 2008
Why regional markets matter: Driving Distance and Times. Chicago's 35 minutes closer than Kansas City
| From Goreville, IL | Miles | Hours:Minutes | |
| Paducah, KY | 45 | 0:50:00 | |
| St. Louis, MO | 131 | 2:08:00 | |
| Evansville, IN | 144 | 2:21:00 | |
| Nashville, TN | 180 | 2:50:00 | |
| Springfield, IL | 201 | 3:20:00 | |
| Memphis, TN | 205 | 3:00:00 | |
| Louisville, KY | 232 | 3:33:00 | |
| Chicago, IL | 332 | 5:18:00 | |
| Kansas City, MO | 379 | 5:53:00 |
Hi gang, I'll pretty this up at some point but the message is clear: we have export markets which have little to do with upstate. Louisville's about 2 hours closer than Chicago. Nashville, TN is closer than Springfield, IL. Memphis is 2 hours closer than Chicago.
This matters, and I want "regional" to get into the vocabulary ASAP. Geography's a hard thing to grasp, and when any of us (me included!) say Illinois it means a lot of things that I, for one, don't always understand it to mean. The latitude down here is the same as Roanoake VA and San Francisco..... It matters. Thanks.
Yours for good eats,
Mike (Born in Memphis, raised in Grand Tower Illinois, boomerang and proud of it!)
Wednesday, February 20, 2008
Network Based Language Learning: Crowdsource Model
If you can’t manage a trip abroad to learn a foreign language, the Internet and a broadband computer connection may do the job, too.
Commentary: Trend O'Rama!
One of the challenges of learning the foreign language becomes the available time to practice, and to a great extent learning the agility required for fluency.
Over time, both "real" video and improving audio will continue to enhance the "being there" experience (referred to as telepresence). For the shy, virtual worlds (e.g., Second Life) will be used to shield identities (as well as promise French instruction from, say, Pepé Le Pew).
Commentary: Trend O'Rama!
One of the challenges of learning the foreign language becomes the available time to practice, and to a great extent learning the agility required for fluency.
"LiveMocha (livemocha.com), for example, is a free site where members can tackle 160 hours of beginning or intermediate lessons in French, German, Mandarin Chinese, Spanish, Hindi or English. There is no charge for tutoring; instead, members tutor one another, drawing on their expertise in their own native language."The "technology" has great potential (and testimonials from its users) and solves problems of time, location, and costs. Expect with time that these services will have certifications; institutions might well consider how to interoperate with these networked educational programs and leverage the resources offered.
Over time, both "real" video and improving audio will continue to enhance the "being there" experience (referred to as telepresence). For the shy, virtual worlds (e.g., Second Life) will be used to shield identities (as well as promise French instruction from, say, Pepé Le Pew).
Friday, February 15, 2008
Rural Telecommunications: No Dial Tone
During the recent ice storm, turns out that the telco central office to the south of me does not have a permanent power generator (UPS). They had to truck in a generator on a truck. Apparently, people were quite well up the creek; one institution was effectively out of service for two days.
Tuesday, February 12, 2008
Bilbo! 'sup?
Just got power back here (ice storm, my woods went going off like rifle shots last night, my 125 pound dog skidded off front porch, and just got coffee at noon here).
Client to my south is MIA: the entire chunk of the state 10 miles south of me still goes without power. Could be a ton worse, I know.
It’s Tuesday, and I may have the truck chipped open by Thursday.
Regarding Virtual Tape I'm trying to find some kind of angle on it that hasn't been taken in before. Going back to some stuff I wrote earlier, I don't know that anyone would go into a tape environment if they didn't have existing backup software that expected to work with tape. Legacy apps hold back some of the transition.
The industry seems to consolidate towards Disk To Disk To Tape now; I'm trying to get to some true Disk Wallah friends to get better interpretation of that particular issue *but* seems that at least with that middle tier of disk I may be able to do more of the dedupe optimization because I don't have to muck with the serial access of the tape media.
I don't want to proclaim tape is dead, but it surely seems, from reading the "talking heads" usual suspects out there in the analyst community that it has really transformed: what matters is the virtual, not the tape. Still smelling intrinsic lock in, (so what, like, get a manager application to manage the manager manager application?) and there are many extant rants about the "who runs this" and unhappy back doors into the system to enable features like "if 2+2=5 then 2+2=5" kinds of kludges.
"One Ring to rule them all, One Ring to find them, One Ring to bring them all and in the darkness bind them" as Tolkien wrote.
I'm all over the caveats for the emptors, and suggest a vigorous scrubbing of any proposal to ensure that total costs and operational implications are walked through in detail. Something has to be "running" the infrastructure with assurance of the integrity of the data. De Dupe could be the planner who really doesn't game it out. I, truly, glaze over and turn to the serious people with scars and pocket protectors. My sense though is that tape fades: spinning moves to solid state, and even solid state becomes ephemeral with nano and finer grained media. The Virtual Tape business has saved millions of dollars (dozens of Loonies at today's rate) and solves a serious problem by abstracting the media, to a great extent, from the implementation. But a full model of "the business" can work wonders.
Think “The Sims” for Storage.
So maybe I should just post this? Another tree broke, dog is needy.
Client to my south is MIA: the entire chunk of the state 10 miles south of me still goes without power. Could be a ton worse, I know.
It’s Tuesday, and I may have the truck chipped open by Thursday.
Regarding Virtual Tape I'm trying to find some kind of angle on it that hasn't been taken in before. Going back to some stuff I wrote earlier, I don't know that anyone would go into a tape environment if they didn't have existing backup software that expected to work with tape. Legacy apps hold back some of the transition.
The industry seems to consolidate towards Disk To Disk To Tape now; I'm trying to get to some true Disk Wallah friends to get better interpretation of that particular issue *but* seems that at least with that middle tier of disk I may be able to do more of the dedupe optimization because I don't have to muck with the serial access of the tape media.
I don't want to proclaim tape is dead, but it surely seems, from reading the "talking heads" usual suspects out there in the analyst community that it has really transformed: what matters is the virtual, not the tape. Still smelling intrinsic lock in, (so what, like, get a manager application to manage the manager manager application?) and there are many extant rants about the "who runs this" and unhappy back doors into the system to enable features like "if 2+2=5 then 2+2=5" kinds of kludges.
"One Ring to rule them all, One Ring to find them, One Ring to bring them all and in the darkness bind them" as Tolkien wrote.
I'm all over the caveats for the emptors, and suggest a vigorous scrubbing of any proposal to ensure that total costs and operational implications are walked through in detail. Something has to be "running" the infrastructure with assurance of the integrity of the data. De Dupe could be the planner who really doesn't game it out. I, truly, glaze over and turn to the serious people with scars and pocket protectors. My sense though is that tape fades: spinning moves to solid state, and even solid state becomes ephemeral with nano and finer grained media. The Virtual Tape business has saved millions of dollars (dozens of Loonies at today's rate) and solves a serious problem by abstracting the media, to a great extent, from the implementation. But a full model of "the business" can work wonders.
Think “The Sims” for Storage.
So maybe I should just post this? Another tree broke, dog is needy.
Saturday, February 09, 2008
Failed US Rural Broadband Policy
I posted a shorter version of this comment at the Chronicle of Higher Education in response to their article "Government Report Lauds Broadband Progress."
The two reports discussed are:
The two reports discussed are:
- “Networked Nation: Broadband in American, 2007” (pdf) by the National Telecommunications And Information Administration (NTIA) and
- "A Blueprint For Big Broadband"presented by EDUCAUSE at the recent "State of The Net" Congressional Conference
Poor, Known Faulty Sample Method Used
The NTIA report continues to rely upon illogical survey information for broadband: five digit zip codes.
In rural areas, some zip codes cover large areas, but if the respondent at the edge of a city with broadband can say “yep, I got broadband,” that entire zip code counts as having broadband service.
This sampling defect is well known and has been a point of annoyance for policy makers who understand the desire to game the system.
US Rural: Slow Deployment, Low Penetration, Stifled Innovation
With reference to regional and rural economic development, educational facilities here (in Southern Illinois) quickly find the limitations of broadband infrastructure. It’s minimal, and localized, at best, and expectations have been worn down by the incumbents.
Rural broadband is essential to sustainable, self sufficient, United States economies. Not sufficient, but certainly necessary.
This NTIA report will, unfortunately, be used as a rebuttal to those trying to make for rural change.
Those who tout its statistics should note that it is a lampoon of good policy, the data are blurred, and the myth of “competitive market solutions” continue apace.
The changes are coming, but the innovation seems to come from upstarts; the incumbent providers apparently move only when threatened.
Educause Report Substantiates Failed US Policy
EDUCAUSE raises good points vis a vis relative US position, but the emphasis (from my own self interest!) is not so much the 100Mb services as the need to get deployments of above 1Mb services, at a minimum, into the “flyover country” and economically depressed towns.
Netflix, for example, needs at least 1.0 Mb for good video quality, with best quality at 1.5+ Mb services.
But the use of a network adds value to all the connected.
These higher speeds will enable new educational models, new business forms, and new sources of entertainment on demand. Applications (payroll, hr, product catalogs, customer relationship information, health records) are becoming more a Service In The Cloud, and designers are improving the effectiveness of "local" and "distant" cooperative applications.
A small business can deliver much of its own infrastructure as a service reached across a reliable, high capacity, network.
Apple continues to drive innovation in the educational segment: iTunes U delivers digital content for free to students from Kindergarten and up. Apple provides free materials for "how to do this" type of education. But this all depends upon a robust ubiquitous broadband network into the communities served.
And we in the rural parts of the world haven't got that network yet, although this was promised in deals made back in the mid 1990s in exchange for "deregulation".
Poppycock.
And the network latency of many "well you could do this" proposed solutions of EDGE, satellite, etc. is a fable best told to the illiterate.
Of course, the further parts of the guile includes capacity lids for numbers of bits passed through the network to "protect the infrastructure". Balderdash.
Market Failure
Because of the low population density of the rural US, providers using old school thinking and relying upon old economic models give a great example of “market failure”; precisely the sorts of conditions which drove rural electrification and taxes for “Universal Service” for the regulated Bell monopoly.
The relief may well come from initiatives that resemble the TVA/REA works and rural electric coops. By other measures in the news these days, history seems to be repeating itself in other ways as well.
Nonetheless, when my neighbor's copper wire from the road to the house broke, the local telco rolled out a truck and crew to replace the copper wire with.... more copper wire. Three times. Not the crew's fault, but it is a grand example of failed policy. Give those telcos out here the Hobgoblin award.
Tuesday, January 15, 2008
Virtualization, Consumers, Networks, No Spin
A small company of 9 people called 37signals.com has a base of over 2,000,000 customers using one of their lead collaborative products. The company's excelled at "Goldilocks" design:not burdening software products with too much function.
From their blog at http://www.37signals.com/svn/posts/749-ask-37signals-numbers
Overall storage stats (Nov 2007)
They're going Xen, and reducing the number of servers by a little more than half.
What's more: they're a poster child for Amazon Simple Storage Service (Amazon S3).
Now, there's Google moving in this storage space, plus new players in the scrum, such as Nirvanix, and the Cleversafe, and who knows what that could be spun out from projects like (US) National LambdaRail.
Consumers
For a current project, looking to use "in the cloud" storage to host multimedia consumer "objects" and also musing upon the potential of little gadgets like iPod Touch to hold 10 hours of video with a price tag for the "1.0" of $300 US (which is, I think, about 80 Loonies this week). Since there's going to be a goat rodeo in producing content and control mechanisims for versioning and etc..... it becomes likely that we'll serve the content itself on an external network.
I'm lobbying for the Storage as a Super Hierarchy (wait for it: StaaSH) involving nothin' but net.
Life's too short.
Virtualization
Meanwhile, I keep looking at virtualization, and virtual tape. Spoke with a friend or two, and the interactivity of virtualization and de dupe starts to get interesting. I'm also of the opinion that no one would by choice run down the road of short tape for their application given the (wave hands) shifting cost curves of spinning drives, and especially considering the growth of solid state drives (I know, I know....) but in recognition also of nano tech showing up in what now..... 14 months.
So, my conjecture is that if one approached the problem set as a greenfield exercise, the pressure's very much in favor of non removable, and probably, non spinning media sooner rather than later. I'd be very interested in anyone's "fully burdened" cost analysis of this tech over time.
We used to have pizza boxes and pickles in the data center.....
No Spin
From their blog at http://www.37signals.com/svn/posts/749-ask-37signals-numbers
Overall storage stats (Nov 2007)
- 5.9 terabytes of customer-uploaded files
- 888 GB files uploaded (900,000 requests)
- 2 TB files downloaded (8,500,000 requests)
They're going Xen, and reducing the number of servers by a little more than half.
What's more: they're a poster child for Amazon Simple Storage Service (Amazon S3).
Now, there's Google moving in this storage space, plus new players in the scrum, such as Nirvanix, and the Cleversafe, and who knows what that could be spun out from projects like (US) National LambdaRail.
Consumers
For a current project, looking to use "in the cloud" storage to host multimedia consumer "objects" and also musing upon the potential of little gadgets like iPod Touch to hold 10 hours of video with a price tag for the "1.0" of $300 US (which is, I think, about 80 Loonies this week). Since there's going to be a goat rodeo in producing content and control mechanisims for versioning and etc..... it becomes likely that we'll serve the content itself on an external network.
I'm lobbying for the Storage as a Super Hierarchy (wait for it: StaaSH) involving nothin' but net.
Life's too short.
Virtualization
Meanwhile, I keep looking at virtualization, and virtual tape. Spoke with a friend or two, and the interactivity of virtualization and de dupe starts to get interesting. I'm also of the opinion that no one would by choice run down the road of short tape for their application given the (wave hands) shifting cost curves of spinning drives, and especially considering the growth of solid state drives (I know, I know....) but in recognition also of nano tech showing up in what now..... 14 months.
So, my conjecture is that if one approached the problem set as a greenfield exercise, the pressure's very much in favor of non removable, and probably, non spinning media sooner rather than later. I'd be very interested in anyone's "fully burdened" cost analysis of this tech over time.
We used to have pizza boxes and pickles in the data center.....
No Spin
Sunday, January 06, 2008
Subscribe to:
Posts
(
Atom
)