Just a lead in: cloud computing will be vital for "small" businesses because "a man's got to know his limitations."
Now we might twitter, and for all that companies growing from startup to second stage, usually around 5+ employees, all at once notice a rumble in the fuselage.
We had service bureaus, used ADP for payroll, and edge into Quicken to run the books. Cloud computing, next big thing (and has been for years).
Showing posts with label SanNasTimes Article. Show all posts
Showing posts with label SanNasTimes Article. Show all posts
Monday, October 13, 2008
Tuesday, August 12, 2008
Disaster. Recovery. Invention.
In most of this last year's pieces here at the land of SANNAS (a fine team of wonks and also a fantastic snack with a good lager) the theme often centered upon increasing demands of distributed ephemeral data and the challenge of managing the process of custody and validation.
This article's being typed into a 1Gb stick; about 2Mb of that stick contains an encryption program; acceptable overhead IMHO for the promise of securing the Next Great Novel and Sudoku downloads, as well as the launch codes for the Acme ® ‘Lil Armageddon family of products, my sonnets to Paris Hilton and other juicy bits.
I do not, as they say, keep a tidy desk. My brain stays healthy by understanding my own LIFO filing system and an ability to understand the strata and the high probability parts of the piles wherein nestles the Airline Magazine or the clipping of a local paper's crank I wanted to riff upon at leisure. This represents an elegant strategy promoting mental health albeit with a risk of structural collapse of the entropy-friendly piles of the arcane lore.
Somewhere, someone must be working on a desktop computing metaphor that allows for significant standing loads. Bearing walls. Like that. At the very least, maybe something like "The Clapper" to find that 1Gb slice of memory...
So, here's the thing: data all over the place, connected and unconnected with the not so subtle growth of metadata to describe the context and provenance of information along with the burden of incremental data to manage the data and thereby the added processing cycles for data management itself. Extremely bright designers have delivered high value tool infrastructures, and I, for one, am not worthy of holding their pocket protectors in the area of difficult code and algorithm implementation, and generally customer focused implementations.
But in the realm of Disaster Recovery mechanisms and services, preemptive trumps reactive. Some scenarios of the mode of disaster, use cases, deliver an example.
Pandemic Flu, Weather, Earthquake, Toxic Spill, extended outages of power, water, other broken infrastructure should be the object of sandtable exercises, at a minimum, to game through what (might likely) work in these scenarios.
Rather makes removable media a bit of a problem during times of "saw fan, engaged same", not to mention getting to the unnetworked unautomated and unavailable mélange of annotated manuals and Post It notes which, don't you know, are the keys to the kingdom, whether one acknowledges that or not.
The adhocracy of portable data (iPhone, et.al.)seems to drive the industry towards some sort of nexus, wherein the overall practice and form of storage management and optimization will trend toward something that looks very much like Open Source toolkits and standards. For some this will be the defining disaster; however, other mature technology (e.g., MVS et seq) informs us that the core functionality and benefits of the "mature" technology do not by any means always disappear, but become the subject of new core businesses and invention. Ye Olde Virtual Machine has shown a tenacity in meeting the market need, albeit in quite new forms.
So, vis a vis Disaster Recovery, the pressure is on for shifts that make for highly interoperable and fungible networked storage resources (think Googleplex) with arbitrarily attached processing and storage tools. A lot of the "math" to predict the future comes from the good works of people like Gene Amdahl and Jim Gray (of Microsoft fame) in that a feasibility test can be accomplished with relative ease; with new cost factors and performance factors in hand, the maxim of "in the long run, all costs are variable" will again prove in with new invention. Of particular interest will be the results of open standards initiatives (akin to Web 3.0 posited mechanisms)where ontology will bloom like kudzu in Dixie.
And that, as the young lady informs us, is "hot".
This article's being typed into a 1Gb stick; about 2Mb of that stick contains an encryption program; acceptable overhead IMHO for the promise of securing the Next Great Novel and Sudoku downloads, as well as the launch codes for the Acme ® ‘Lil Armageddon family of products, my sonnets to Paris Hilton and other juicy bits.
I do not, as they say, keep a tidy desk. My brain stays healthy by understanding my own LIFO filing system and an ability to understand the strata and the high probability parts of the piles wherein nestles the Airline Magazine or the clipping of a local paper's crank I wanted to riff upon at leisure. This represents an elegant strategy promoting mental health albeit with a risk of structural collapse of the entropy-friendly piles of the arcane lore.
Somewhere, someone must be working on a desktop computing metaphor that allows for significant standing loads. Bearing walls. Like that. At the very least, maybe something like "The Clapper" to find that 1Gb slice of memory...
So, here's the thing: data all over the place, connected and unconnected with the not so subtle growth of metadata to describe the context and provenance of information along with the burden of incremental data to manage the data and thereby the added processing cycles for data management itself. Extremely bright designers have delivered high value tool infrastructures, and I, for one, am not worthy of holding their pocket protectors in the area of difficult code and algorithm implementation, and generally customer focused implementations.
But in the realm of Disaster Recovery mechanisms and services, preemptive trumps reactive. Some scenarios of the mode of disaster, use cases, deliver an example.
Pandemic Flu, Weather, Earthquake, Toxic Spill, extended outages of power, water, other broken infrastructure should be the object of sandtable exercises, at a minimum, to game through what (might likely) work in these scenarios.
Rather makes removable media a bit of a problem during times of "saw fan, engaged same", not to mention getting to the unnetworked unautomated and unavailable mélange of annotated manuals and Post It notes which, don't you know, are the keys to the kingdom, whether one acknowledges that or not.
The adhocracy of portable data (iPhone, et.al.)seems to drive the industry towards some sort of nexus, wherein the overall practice and form of storage management and optimization will trend toward something that looks very much like Open Source toolkits and standards. For some this will be the defining disaster; however, other mature technology (e.g., MVS et seq) informs us that the core functionality and benefits of the "mature" technology do not by any means always disappear, but become the subject of new core businesses and invention. Ye Olde Virtual Machine has shown a tenacity in meeting the market need, albeit in quite new forms.
So, vis a vis Disaster Recovery, the pressure is on for shifts that make for highly interoperable and fungible networked storage resources (think Googleplex) with arbitrarily attached processing and storage tools. A lot of the "math" to predict the future comes from the good works of people like Gene Amdahl and Jim Gray (of Microsoft fame) in that a feasibility test can be accomplished with relative ease; with new cost factors and performance factors in hand, the maxim of "in the long run, all costs are variable" will again prove in with new invention. Of particular interest will be the results of open standards initiatives (akin to Web 3.0 posited mechanisms)where ontology will bloom like kudzu in Dixie.
And that, as the young lady informs us, is "hot".
Thursday, July 17, 2008
Disk Payload Management
Transfer of data has an upper bound of the speed of light and a lower bound of amount of a budget, excluding strange action at a distance and physics not yet known. It's all fun and games until something divides by zero.
In a delightful teaser article, Neil J. Gunther's "The Guerrilla Manual" delivers a bolus of refreshing views on capacity planning and performance management with a cleansing amount of terse common sense.
In particular, he notes, "You never remove the bottleneck, you just shuffle the deck."
Network Effects and Blinkenlights
Back in the mid 1980s, at least one large financial institution allocated IT budgets using a simple ratio of numbers of customer accounts by type, with appropriate finagle factors. At least it was a model that, assuming a lot of linearity, had simplicity and apparent transparency going for it.
Of course, these were the times of data centers with big boxes, and the occasional minicomputer. The unit costs of processing, networks, and storage were significant vis a vis cycles or bits or bytes per dollar and cycles per watt.
Of course, also, the use cases for the technology moved rather slowly, with occasional punctuation with growing online inquiry from, say, customer service agents or the addition of Automatic Teller Machines to the CICS olio of the big iron code.
More gadgets and new approaches to programming by the end users (unclean!!!) resulted in rather surprising effects upon infrastructure through rampant flaming queries (what did he say?) and even complete suites of large scale computing systems dedicated to new types of models. In the case of financial services, one big dude jammed with APL for determination of fixed income dynamics. APL, for those who don't recall, was developed for passive aggressive savants who didn't want management looking into what they'd written. But, with letting the punishment fit the crime, APL rocked for matrix operations and was a darling of the early generation of quants, including those laugh a minute actuaries.
Somewhere, someplace, someone is hacking FPGAs to stick into the Beowulf cluster of X Boxes. I gotta feeling.
So where were we... Oh, so the point is that the common factor around these early instances of "end user" computing involved moderate and increasing network effects. Transactional data could be used as feeds to these user managed systems, and network effects with emphasis upon storage and I/O tuning became significant as a means of moving the bottleneck back to the cpu. Now pick another card.
The disk to disk discussion comprises several use cases, ranging from performance optimization (e.g, put the top 10 movies on the edge of the network) to business continuance to the meta issue of secure transfer and "lockup" of the data. Problem is, how does one deal with this mess which embraces Service Oriented Architectures and Halo dynamism?
Intelligent Payloads?
This problem of placing data and copies of data in "good enough" sites on the network seems encumbered by how these data are tagged in such a way as to inform the "system" itself on the history of the atomic piece of interest as it transits other systems and networks. Perhaps something that appends usage information to the information itself, rather like appending travel stickers to an old steamer trunk tracing its owner's tours of Alice Springs, Kenosha, and Banff.
And no, I'm not advocating still another inband system monitor... more MIBs than MIPS and all of that problem.
This could, I believe, be a fertile area for new types of automation that begin to apply optimization (satisficing, most likely, in the sense of "good enough" strategies, see Herbert Simon for more G2) thereby, maybe (he qualifies again!) to reduce the amount of time and money spent upon forensics and weird extraction of information needed to govern surprisingly fluid dynamic systems.
Zipf's Law (think top 10 lists, 80/20 rule, The Long Tail issues, etc.) and other power law behaviors will still apply to the end product of such analysis, but perhaps the informed payloads will ease the analysts' management of these turbulent parcels. (Some insights to the framing of the problem of getting top level insight into systems structures and how they express emergent behaviors can be found at the Santa Fe Institute and their many papers on "Small World" problems.)
So, the bounds on this problem of course reduces to time and money. That topic also is taken up by Gunther, with emphasis upon what some of my old gang at the Wall Street joint referred to as "the giggle test" for feasibility.
This is a brief piece about an intriguing problem where more insight can be gained from Operations Research methodologies than from Information Technology praxis per se.
It nets out to (sorry) not only if it is not measured, it isn't managed, but add to that the cautionary insight of "if it isn't modeled, it isn't managed."
In a delightful teaser article, Neil J. Gunther's "The Guerrilla Manual" delivers a bolus of refreshing views on capacity planning and performance management with a cleansing amount of terse common sense.
In particular, he notes, "You never remove the bottleneck, you just shuffle the deck."
Network Effects and Blinkenlights
Back in the mid 1980s, at least one large financial institution allocated IT budgets using a simple ratio of numbers of customer accounts by type, with appropriate finagle factors. At least it was a model that, assuming a lot of linearity, had simplicity and apparent transparency going for it.
Of course, these were the times of data centers with big boxes, and the occasional minicomputer. The unit costs of processing, networks, and storage were significant vis a vis cycles or bits or bytes per dollar and cycles per watt.
Of course, also, the use cases for the technology moved rather slowly, with occasional punctuation with growing online inquiry from, say, customer service agents or the addition of Automatic Teller Machines to the CICS olio of the big iron code.
More gadgets and new approaches to programming by the end users (unclean!!!) resulted in rather surprising effects upon infrastructure through rampant flaming queries (what did he say?) and even complete suites of large scale computing systems dedicated to new types of models. In the case of financial services, one big dude jammed with APL for determination of fixed income dynamics. APL, for those who don't recall, was developed for passive aggressive savants who didn't want management looking into what they'd written. But, with letting the punishment fit the crime, APL rocked for matrix operations and was a darling of the early generation of quants, including those laugh a minute actuaries.
Somewhere, someplace, someone is hacking FPGAs to stick into the Beowulf cluster of X Boxes. I gotta feeling.
So where were we... Oh, so the point is that the common factor around these early instances of "end user" computing involved moderate and increasing network effects. Transactional data could be used as feeds to these user managed systems, and network effects with emphasis upon storage and I/O tuning became significant as a means of moving the bottleneck back to the cpu. Now pick another card.
The disk to disk discussion comprises several use cases, ranging from performance optimization (e.g, put the top 10 movies on the edge of the network) to business continuance to the meta issue of secure transfer and "lockup" of the data. Problem is, how does one deal with this mess which embraces Service Oriented Architectures and Halo dynamism?
Intelligent Payloads?
This problem of placing data and copies of data in "good enough" sites on the network seems encumbered by how these data are tagged in such a way as to inform the "system" itself on the history of the atomic piece of interest as it transits other systems and networks. Perhaps something that appends usage information to the information itself, rather like appending travel stickers to an old steamer trunk tracing its owner's tours of Alice Springs, Kenosha, and Banff.
And no, I'm not advocating still another inband system monitor... more MIBs than MIPS and all of that problem.
This could, I believe, be a fertile area for new types of automation that begin to apply optimization (satisficing, most likely, in the sense of "good enough" strategies, see Herbert Simon for more G2) thereby, maybe (he qualifies again!) to reduce the amount of time and money spent upon forensics and weird extraction of information needed to govern surprisingly fluid dynamic systems.
Zipf's Law (think top 10 lists, 80/20 rule, The Long Tail issues, etc.) and other power law behaviors will still apply to the end product of such analysis, but perhaps the informed payloads will ease the analysts' management of these turbulent parcels. (Some insights to the framing of the problem of getting top level insight into systems structures and how they express emergent behaviors can be found at the Santa Fe Institute and their many papers on "Small World" problems.)
So, the bounds on this problem of course reduces to time and money. That topic also is taken up by Gunther, with emphasis upon what some of my old gang at the Wall Street joint referred to as "the giggle test" for feasibility.
This is a brief piece about an intriguing problem where more insight can be gained from Operations Research methodologies than from Information Technology praxis per se.
It nets out to (sorry) not only if it is not measured, it isn't managed, but add to that the cautionary insight of "if it isn't modeled, it isn't managed."
Friday, June 13, 2008
Trusted Infrastructure
Every day someone (or... something) with 218 dot et dot cet dot era out of China intently checks my TCP port connectivity. Relentless, time after time, again and again it seeks what I do not know, just that it signifies.
Cut us a break? Not likely.
Good old 218 (we're on a first octet basis) and his kin know that the herd, large and lacking vaccine, eases the pursuit of new zombies and kindred grift keep those who are far, far, more than "script kiddies" hard at work.
More than to get through spam filters (v149ra, 'frinstance or his sister of ill repute, C. Alice) or help to facilitate commercial dealings involving so often the sad demise of the spouse, uncle, client, former ambassador... tragedies all. I never knew how much wealth sloshes around as the result of corpses. Ambulance chasing seems quite profitable.
"Make Money Fast" now like a Madeline evokes the time and place of a 14.4 "golden age" before a simple "delete" command would be replaced with protocols that put a Level 4 Biohazard facility to shame (and this hinkey mess o code just for a desktop!)
Nothing to see here, move along!
Richard A. Clarke, of US National Security fame, describes in his new book "Your Government Has Failed You", the results of a security exercise with Department of Defense systems. White hats commenced to do that hoo doo that they do so well... and pretty much it was "game over" in a trice: penetration aptly describes what happened.
The guvment got Intrusion monitors installed. They lit up like Bay Ridge Brooklyn on Christmas Eve with all sorts of dubious packets from UnKn0wn U53r types. Clarke quotes managers telling him that until the hoods started setting off the intrusion alarms there had never been any putative bad guys. Somehow the security tech now attracted the bums, etc. Familiar story that I'm inclined to believe.
He also advocates a partitioned Internet, hardening part of the services for secure transactions and communications.
I've been there philosophically myself; and nonetheless am avidly in favor of an unrestrained Internet as well. No hobgoblins here.
The company I was with in 2001/2002 had a trusted client server implementation that did very serious authentication (non repudiation, yadda yadda rather like Moody Blues lyrics, those shalts and shants in them there specs ballyhoo optional). Pretty much good 2 go, add Series A and shake. Well, we had the hip but not the audience.
But the big deal was that our technology needed a pretty well suborned geek to break it (it wasn't open source, and in that I now trust it less but I'm just sayin'). It kept things nice and shiny and very private as an overlay to the Big I at large. Other Big Problem though comes from oceans of distributed data.
In a world: Queue Don LaFontaine
In a world where everything moves to an emergent ad hoc architecture of networked promiscuous storage services, where devices (picture frame, 1Gb wrist bracelet, virtualized server storage complexes, distributed p2p/p4p, DRM, TiVo, and even instrumentation motes reporting on movement of RFID tags and small area hydrology and nitrogen measures, etc.etc. containing massive amounts of data) often in surprisingly readable formats.
In a world... where consumers have started to put health records within the uberplexes of Microsoft and Google (getting the providers en masse to adopt it, different problem) and the concept of insurance moves from shared risk to, well, greed seems a good word.
In a world... where the laptops of one government are targeted and tossed (usage as in noir cant) for records of another government's dissidents and the bon mots of "gentlemen do not read each others mail" have no standing...
In a world where the retiring Secretary of Health and Human Services says "For the life of me, I cannot understand why the terrorists have not attacked our food supply because it is so easy to do." (NYTimes, December 2004)
In a world where a friend takes an extended business trip, returns to find that his kids have made five movies and posted them on YouTube from his Mac.
In a world where cameras come free with the printer, and a postage stamp of silicon tells me I'm on shot one of eighteen hundred.
Trusted Infrastructure Elements
So, here's where it goes, maybe. I have two things: things one and thing two.
Ubiquitous One Time Pads: All private, all the time.
Perhaps standing the security and privacy on end makes sense. Processing cycles and (good grief I actually typed core but stet!) core become pretty inexpensive, so running an algorithm to encode everything might not be a bad way to go. Heck, with some of the new word processing file formats alleging "open" protocols we seem to be half way there already; I can't open the attachments. We already see the unit costs of solid state "drives" precipitously dropping; headroom for small processing to encrypt onchip data could be a value added feature. The great thing is that these solid state devices are so small and hold so much data. The problem is that these solid state devices are so small and hold so much data.
Distributed Spread Spectrum Storage Service
Pseudorandom multi channel information paths, with a "big enough" local store to allow for unreliable networked service. The signal (information) hides in the noise. A form of this unreliable network appears as "The Byzantine Generals" problem, specifically Byzantine fault tolerance, wherein unreliable communications mechanisms become designed "around". Reason for this part of the puzzle is that there's so much data in so many places (cheap, dense, ubiquitous) that the use case for managed services around security and recovery needs to account for mobility, and placement of user data in many places. Key point though, is that a number of public and private researchers have been going at the reliability from unreliable infrastructure for a good while now (like, even ARPANET embodied this meme) and those wacky folks at Bittorrent (for example) have large amounts of distributed secure services in their beginning gene pool plus the understanding of how to reliably place data all over whilst maintaining service levels. If the Bit folks seem too edgy, then take a look at the Akamai technology model, and throw in a dose of Meraki Networks or perhaps Spain's FON.
Point is, seems that "the industry" has a pretty solid base of knowledge in sundry places that can be applied to the secure distributed data problem, and that the notional risks to private data help to monetize the innovation's value. The architects/designers Ray and Charles Eames explored the concept of "kit of parts" for building; I think that much of that "kit of parts" exists in the public domain and will be amended with patent expiries over the next several years.
Your Mileage May Vary
That vision above, if it merits that moniker, ignores some Big Problems like distributed key management for starters, twenty volumes of USENET rants about Pretty Good Privacy (PGP), in band versus out of band control, and a whole lot of other things. I believe the joke ends with the physicist saying "we've solved the problem for a sphere." Managing distributed data "in a world" of ubiquitous storage seems the next grand challenge.
Good hunting.
Cut us a break? Not likely.
Good old 218 (we're on a first octet basis) and his kin know that the herd, large and lacking vaccine, eases the pursuit of new zombies and kindred grift keep those who are far, far, more than "script kiddies" hard at work.
More than to get through spam filters (v149ra, 'frinstance or his sister of ill repute, C. Alice) or help to facilitate commercial dealings involving so often the sad demise of the spouse, uncle, client, former ambassador... tragedies all. I never knew how much wealth sloshes around as the result of corpses. Ambulance chasing seems quite profitable.
"Make Money Fast" now like a Madeline evokes the time and place of a 14.4 "golden age" before a simple "delete" command would be replaced with protocols that put a Level 4 Biohazard facility to shame (and this hinkey mess o code just for a desktop!)
Nothing to see here, move along!
Richard A. Clarke, of US National Security fame, describes in his new book "Your Government Has Failed You", the results of a security exercise with Department of Defense systems. White hats commenced to do that hoo doo that they do so well... and pretty much it was "game over" in a trice: penetration aptly describes what happened.
The guvment got Intrusion monitors installed. They lit up like Bay Ridge Brooklyn on Christmas Eve with all sorts of dubious packets from UnKn0wn U53r types. Clarke quotes managers telling him that until the hoods started setting off the intrusion alarms there had never been any putative bad guys. Somehow the security tech now attracted the bums, etc. Familiar story that I'm inclined to believe.
He also advocates a partitioned Internet, hardening part of the services for secure transactions and communications.
I've been there philosophically myself; and nonetheless am avidly in favor of an unrestrained Internet as well. No hobgoblins here.
The company I was with in 2001/2002 had a trusted client server implementation that did very serious authentication (non repudiation, yadda yadda rather like Moody Blues lyrics, those shalts and shants in them there specs ballyhoo optional). Pretty much good 2 go, add Series A and shake. Well, we had the hip but not the audience.
But the big deal was that our technology needed a pretty well suborned geek to break it (it wasn't open source, and in that I now trust it less but I'm just sayin'). It kept things nice and shiny and very private as an overlay to the Big I at large. Other Big Problem though comes from oceans of distributed data.
In a world: Queue Don LaFontaine
In a world where everything moves to an emergent ad hoc architecture of networked promiscuous storage services, where devices (picture frame, 1Gb wrist bracelet, virtualized server storage complexes, distributed p2p/p4p, DRM, TiVo, and even instrumentation motes reporting on movement of RFID tags and small area hydrology and nitrogen measures, etc.etc. containing massive amounts of data) often in surprisingly readable formats.
In a world... where consumers have started to put health records within the uberplexes of Microsoft and Google (getting the providers en masse to adopt it, different problem) and the concept of insurance moves from shared risk to, well, greed seems a good word.
In a world... where the laptops of one government are targeted and tossed (usage as in noir cant) for records of another government's dissidents and the bon mots of "gentlemen do not read each others mail" have no standing...
In a world where the retiring Secretary of Health and Human Services says "For the life of me, I cannot understand why the terrorists have not attacked our food supply because it is so easy to do." (NYTimes, December 2004)
In a world where a friend takes an extended business trip, returns to find that his kids have made five movies and posted them on YouTube from his Mac.
In a world where cameras come free with the printer, and a postage stamp of silicon tells me I'm on shot one of eighteen hundred.
Trusted Infrastructure Elements
So, here's where it goes, maybe. I have two things: things one and thing two.
Ubiquitous One Time Pads: All private, all the time.
Perhaps standing the security and privacy on end makes sense. Processing cycles and (good grief I actually typed core but stet!) core become pretty inexpensive, so running an algorithm to encode everything might not be a bad way to go. Heck, with some of the new word processing file formats alleging "open" protocols we seem to be half way there already; I can't open the attachments. We already see the unit costs of solid state "drives" precipitously dropping; headroom for small processing to encrypt onchip data could be a value added feature. The great thing is that these solid state devices are so small and hold so much data. The problem is that these solid state devices are so small and hold so much data.
Distributed Spread Spectrum Storage Service
Pseudorandom multi channel information paths, with a "big enough" local store to allow for unreliable networked service. The signal (information) hides in the noise. A form of this unreliable network appears as "The Byzantine Generals" problem, specifically Byzantine fault tolerance, wherein unreliable communications mechanisms become designed "around". Reason for this part of the puzzle is that there's so much data in so many places (cheap, dense, ubiquitous) that the use case for managed services around security and recovery needs to account for mobility, and placement of user data in many places. Key point though, is that a number of public and private researchers have been going at the reliability from unreliable infrastructure for a good while now (like, even ARPANET embodied this meme) and those wacky folks at Bittorrent (for example) have large amounts of distributed secure services in their beginning gene pool plus the understanding of how to reliably place data all over whilst maintaining service levels. If the Bit folks seem too edgy, then take a look at the Akamai technology model, and throw in a dose of Meraki Networks or perhaps Spain's FON.
Point is, seems that "the industry" has a pretty solid base of knowledge in sundry places that can be applied to the secure distributed data problem, and that the notional risks to private data help to monetize the innovation's value. The architects/designers Ray and Charles Eames explored the concept of "kit of parts" for building; I think that much of that "kit of parts" exists in the public domain and will be amended with patent expiries over the next several years.
Your Mileage May Vary
That vision above, if it merits that moniker, ignores some Big Problems like distributed key management for starters, twenty volumes of USENET rants about Pretty Good Privacy (PGP), in band versus out of band control, and a whole lot of other things. I believe the joke ends with the physicist saying "we've solved the problem for a sphere." Managing distributed data "in a world" of ubiquitous storage seems the next grand challenge.
Good hunting.
Wednesday, May 14, 2008
Compliance CENTREX
What went around comes around.
Compliance, to pass an examination or be prepared for a contingency, has foundation from commercial good practice and regulatory (including legal) conformance.
Higher level compliance concerns evoke the wisdom of first do no harm, eat green leafy vegetables, and get moderate exercise. Business continuance, or continuance of the entity or ones job, ripens with common sense. The broader topics include fiduciary stewardship, management of proprietary internal information, and of client, patient, supplier, et.al. privacy, and, fundamentally, the determination of trust.
In forming the deeply connected systems involving the customers, the organizations, and the suppliers the challenge of determining trust provably is, well, not easy.
In a recent national survey, respondents uniformly agreed that this validation of a system, the compliance issues in managing the archiving and retrieval (or demise) of data worked, by degree, well in larger organizations but was woefully lacking in many others.
Be advised that this national survey involved three people I know.
But they were pretty far away from each other.
A couple of common themes though, on the broad area of what's up with that compliance thing?
Small Organizations Getting Crushed
Many smaller organizations encounter compliance, in the broad sense, only after something has gone quite wrong. The data systems come under the management of one "s/he who knows" and typically that s/he who knows avers that all's under control, nothing to see, move along. Quite often, that's a correct assessment but the Black Swan of Something Wicked will assuredly appear like the first tulip of spring.
The many conceivable test cases and demonstrations available to exercise a procedure or software implementation become quickly incomplete; the "many eyeballs" of the Linux community or MSDN or the SAP experts or Cisco herders or System Z mavens or Wii Wunderkind and the rest may already have the fix in hand, but the bolus of healing code remains unknown to the many.
So staying current with what's on with the system in the technology sense underpins meaningful compliance.
The management of even a small system, when networked, overwhelms or at least distracts from the business at hand, the getting the work done part of work. Script kiddies, malware, and zombie makers test test test the borders and their numbers still grow. Firewalls capture the IP of pinging drones with the tireless cold focus of The Terminator.
But enough of the light side.
Somewhere there's a law waiting to bite you.
In addition to the issues of compliance with securing systems from perpetrators and miscreants, somewhere one can find comfort in some legislation from someone who should, but does not, know "better" that will increase costs for compliance if not explicitly be purely intractable by the placement of the liability of non compliance with an impossible dream.
Those public policy decisions might include draconian warnings for browsing a Goya Maja or a 1970s Sears Catalog, and more seriously issues of record retention and access for, among other things, financial and health transactional data, and emails (which increasingly go beyond text and have average object sizes growing like Kudzu). I wonder if Wall Street still drives those tapes around in the vans, and whether aging tape drives live on to read archaic media. This leads to still more policy decisions, internal to an organization and implementations driven by external policy makers.
For the smaller organization, staying on top of the goat rodeo that entails version management of desktop OS and warring tribes of antiwhatever software chewing through silicon to the joy of chipkateers and the anguish of people just trying to get along with recreational and the oh so serious land of business computing, now that goat rodeo builds character in and of itself.
To overlay "unfunded mandates" on the already maxtasked elves of IT, well that's just Bondo on the cake.
Hence: Compliance Centrex
In the dark ages of the 1970s and 1980s, telephone companies in more urban climes implemented a product called Centrex (Centralized Exchange) service.
The mainframe (Central Office) provided features from a bunker managed by truly serious professionals empowered to code the forwarding of phone calls from one line to another, implement restrictions on calling privileges to certain telephone lines (local only, for example) and wield other powers on the users, who enjoyed very reliable consistent service. Each line got its own phone bill. Later generations of the Centrex miracle provided limited administration by the customer.
Point: a small team of skilled people can provide a lot of oomph across a customer base. 37signals with their millions of subscribers being one example already covered here.
Somewhere I hope there's a graduate student looking at the marginal utility of willingness to pay with notional conjoint analysis studies.
More fun than Facebook!
Now, the code to be cracked appeals most to small and medium scale enterprises; bundling policy based compliance features as a software overlay or a true "tin can with widgets + code inside" offers some hope. Think of an XML (etc.) publishing service informing the installed base of gadgetry on the latest rules for archiving, say, emails. Recognize also that the "big" end of the market also may actively seek compliance services: Collaborative Software Initiative's reference implementation for custodial information is but one example.
Towards the Compliance Appliance?
Here in the US, a couple of firms now warrant identity protection with a $1,000,000 guarantee($400 Loonies thereabouts). Will we see participation of new risk instruments, insurance akin to errors and omissions, appearing as part of new "tough" compliance appliances?
Compliance as a service has, I believe, legs in the same sense that an antivirus automatic update does. Some "professional compliance" process exists for food companies: third parties provide certification of the supply chain end of the food pipe to ensure that custodial information, security and hygiene of premises, etc. conform to regulatory standards. Beyond "audit" the service economy for compliance is likely quite amenable to a service provided for fee relationship, increasing service revenues for the storage company (or the applications company) along with driving new purpose into the consigliore side of the bidnez.
Continuous monitoring already available for "tested hacker safe" affirms the underlying business model.
The management of the infrastructure, especially for the smaller companies, takes too many cycles and depends upon people who rarely encounter a Black Swan in their Sarbox, eh?
Teams matter; effective new ones will cross borders of all sorts.
Compliance, to pass an examination or be prepared for a contingency, has foundation from commercial good practice and regulatory (including legal) conformance.
Higher level compliance concerns evoke the wisdom of first do no harm, eat green leafy vegetables, and get moderate exercise. Business continuance, or continuance of the entity or ones job, ripens with common sense. The broader topics include fiduciary stewardship, management of proprietary internal information, and of client, patient, supplier, et.al. privacy, and, fundamentally, the determination of trust.
In forming the deeply connected systems involving the customers, the organizations, and the suppliers the challenge of determining trust provably is, well, not easy.
In a recent national survey, respondents uniformly agreed that this validation of a system, the compliance issues in managing the archiving and retrieval (or demise) of data worked, by degree, well in larger organizations but was woefully lacking in many others.
Be advised that this national survey involved three people I know.
But they were pretty far away from each other.
A couple of common themes though, on the broad area of what's up with that compliance thing?
Small Organizations Getting Crushed
Many smaller organizations encounter compliance, in the broad sense, only after something has gone quite wrong. The data systems come under the management of one "s/he who knows" and typically that s/he who knows avers that all's under control, nothing to see, move along. Quite often, that's a correct assessment but the Black Swan of Something Wicked will assuredly appear like the first tulip of spring.
The many conceivable test cases and demonstrations available to exercise a procedure or software implementation become quickly incomplete; the "many eyeballs" of the Linux community or MSDN or the SAP experts or Cisco herders or System Z mavens or Wii Wunderkind and the rest may already have the fix in hand, but the bolus of healing code remains unknown to the many.
So staying current with what's on with the system in the technology sense underpins meaningful compliance.
The management of even a small system, when networked, overwhelms or at least distracts from the business at hand, the getting the work done part of work. Script kiddies, malware, and zombie makers test test test the borders and their numbers still grow. Firewalls capture the IP of pinging drones with the tireless cold focus of The Terminator.
But enough of the light side.
Somewhere there's a law waiting to bite you.
In addition to the issues of compliance with securing systems from perpetrators and miscreants, somewhere one can find comfort in some legislation from someone who should, but does not, know "better" that will increase costs for compliance if not explicitly be purely intractable by the placement of the liability of non compliance with an impossible dream.
Those public policy decisions might include draconian warnings for browsing a Goya Maja or a 1970s Sears Catalog, and more seriously issues of record retention and access for, among other things, financial and health transactional data, and emails (which increasingly go beyond text and have average object sizes growing like Kudzu). I wonder if Wall Street still drives those tapes around in the vans, and whether aging tape drives live on to read archaic media. This leads to still more policy decisions, internal to an organization and implementations driven by external policy makers.
For the smaller organization, staying on top of the goat rodeo that entails version management of desktop OS and warring tribes of antiwhatever software chewing through silicon to the joy of chipkateers and the anguish of people just trying to get along with recreational and the oh so serious land of business computing, now that goat rodeo builds character in and of itself.
To overlay "unfunded mandates" on the already maxtasked elves of IT, well that's just Bondo on the cake.
Hence: Compliance Centrex
In the dark ages of the 1970s and 1980s, telephone companies in more urban climes implemented a product called Centrex (Centralized Exchange) service.
The mainframe (Central Office) provided features from a bunker managed by truly serious professionals empowered to code the forwarding of phone calls from one line to another, implement restrictions on calling privileges to certain telephone lines (local only, for example) and wield other powers on the users, who enjoyed very reliable consistent service. Each line got its own phone bill. Later generations of the Centrex miracle provided limited administration by the customer.
Point: a small team of skilled people can provide a lot of oomph across a customer base. 37signals with their millions of subscribers being one example already covered here.
Somewhere I hope there's a graduate student looking at the marginal utility of willingness to pay with notional conjoint analysis studies.
More fun than Facebook!
Now, the code to be cracked appeals most to small and medium scale enterprises; bundling policy based compliance features as a software overlay or a true "tin can with widgets + code inside" offers some hope. Think of an XML (etc.) publishing service informing the installed base of gadgetry on the latest rules for archiving, say, emails. Recognize also that the "big" end of the market also may actively seek compliance services: Collaborative Software Initiative's reference implementation for custodial information is but one example.
Towards the Compliance Appliance?
Here in the US, a couple of firms now warrant identity protection with a $1,000,000 guarantee($400 Loonies thereabouts). Will we see participation of new risk instruments, insurance akin to errors and omissions, appearing as part of new "tough" compliance appliances?
Compliance as a service has, I believe, legs in the same sense that an antivirus automatic update does. Some "professional compliance" process exists for food companies: third parties provide certification of the supply chain end of the food pipe to ensure that custodial information, security and hygiene of premises, etc. conform to regulatory standards. Beyond "audit" the service economy for compliance is likely quite amenable to a service provided for fee relationship, increasing service revenues for the storage company (or the applications company) along with driving new purpose into the consigliore side of the bidnez.
Continuous monitoring already available for "tested hacker safe" affirms the underlying business model.
The management of the infrastructure, especially for the smaller companies, takes too many cycles and depends upon people who rarely encounter a Black Swan in their Sarbox, eh?
Teams matter; effective new ones will cross borders of all sorts.
Monday, April 14, 2008
Ephemeral Hardware
Someone, somewhere, once quipped that hardware is out of date software. One can just about take that at face value; Wall Street quants run engines out of FPGA arrays to rapidly calculate banyan fibonacci mojo ratios, let alone the insights given to March Madness here in the United States coming, no doubt, from hijacked cloud computing resources.
Meanwhile, the "not all gigabytes are created equal" applies to this industry where unit costs of storage hardware drop precipitously (and mostly predictably) whilst total storage management costs increase....but at typically lower unit costs.
Then the state diagram of "where does the market find equilibrium" sloshes in another direction with emergent nano densities (we visited that topic in November or December 2007 in here) which sloshes further with announcements of IBM Racetrack topologies (updating previous announcements from neoBUNCH forces on the development of first gen nano which was expected to ship late this year) for chips with billions of atomistic nanowires and even higher densities.
It's interesting to note that Buckminster Fuller, for whom the nanostructures of carbon called "Buckyballs" due to their resemblance to his Geodesic Dome (we've got the one he built and lived in when he taught at the local University here around 37.5W and 89N) also coined the word "ephimeralization".
Do more and more with less and less until you're doing everything with nothing basically sums up that principle; however, what is added to that koan? Haiku? whatever is information.
Seems that, to me, information is what holds that airy ship aloft, information represented as software and rules. So, the perfect storm (strike that, cliché) the fuzzball of storage becomes a witches brew of rapidly increasing "consumer driven" data (oodles of video, oodles of endpoints). And boy howdy, don't them metas get busy?
Problems of managing versions (dedupe, mirrors backups, recoveries), placement of data to manage service levels (cache, multiple copies, P2P or P4P) and (jargon alert) semantic web wherein constructs of tags of tags and winding hyperlinks reflecting states and relationships, not just for catching the great Diet Coke and Mentos research but to get the skinny on what the latest method is to build something, cure something, learn something, share something, price something, buy something, sell something.....
Or use the stored images to catch a thief (or worse) or provide compliance (not just for the SEC; as part of researching video applications for education, a fine gentleman named Mike Fink advised that some hospitals and emergency rooms are adopting video as anticipatory evidence when conducting particularly parlous procedures. (Perspicacity!)
The technology and methods approach not only Kurtzweil's "singularity" of intelligent machines, but the holodeck (hold on, Chief!) appears in the form of a walk about simulation of the hours before Pompeii's demise. The user of the system can move through that simulation by walking. (Heard that one on late night radio just before a discussion of alien abduction but I'm all about eclectic.) This, plus Wii systems have seen rapid adoption in physical therapy....
Consumers all over the joint.
Explicitly: this does not (yet) forecast the immediate demise of spinning media or tape, but the pricing pressures are and will continue to approach "unit costs of zero", with improving wide area connectivity which, while not allowing (for now) ignoring special relativity, multiple managed instances of information and stateful knowledge will appear more and more as abstractions within a deeply interconnected network. Seems not unreasonable to expect some form of bid/ask mechanisms to appear for dynamic allocation of, particularly, entertainment or thematically oriented information linking secure objects, generating additional copies, cleaning out the old data, and optimizing data (and processing) placement based upon dynamic needs.
Given that software itself decays, through maintenance in particular as cited in COCOMO models, etc., the management of storage more becomes the art of software as an increment to the art of understanding the more physical mechanisms of striping, RAID, densities, and mirrors. And the dark force of entropy, lo, will drive more preemptive testing and quality assurance (said the winner of sundry service packs).
And with the flat out impossibility of exhaustively testing every combination of lines of instructions (NP completeness means never having to say you're done) more nuanced and practical methods have emerged to do statistical measures akin to Design Of Experiment methods; however, still better hang on to that last working version.... Given that pretty much every buzzword required in a tech oriented article has been used here (oops, yeah, and Social Networking too!) we might as well toss in quantum computing.
And part of the quantum and string theory models is the many universes theory, which comforts me because it suggests, if true, I might finally be one of the smartest guys in the room, albeit a Cub Reporter.
Happy Spring!
Meanwhile, the "not all gigabytes are created equal" applies to this industry where unit costs of storage hardware drop precipitously (and mostly predictably) whilst total storage management costs increase....but at typically lower unit costs.
Then the state diagram of "where does the market find equilibrium" sloshes in another direction with emergent nano densities (we visited that topic in November or December 2007 in here) which sloshes further with announcements of IBM Racetrack topologies (updating previous announcements from neoBUNCH forces on the development of first gen nano which was expected to ship late this year) for chips with billions of atomistic nanowires and even higher densities.
It's interesting to note that Buckminster Fuller, for whom the nanostructures of carbon called "Buckyballs" due to their resemblance to his Geodesic Dome (we've got the one he built and lived in when he taught at the local University here around 37.5W and 89N) also coined the word "ephimeralization".
Do more and more with less and less until you're doing everything with nothing basically sums up that principle; however, what is added to that koan? Haiku? whatever is information.
Seems that, to me, information is what holds that airy ship aloft, information represented as software and rules. So, the perfect storm (strike that, cliché) the fuzzball of storage becomes a witches brew of rapidly increasing "consumer driven" data (oodles of video, oodles of endpoints). And boy howdy, don't them metas get busy?
Problems of managing versions (dedupe, mirrors backups, recoveries), placement of data to manage service levels (cache, multiple copies, P2P or P4P) and (jargon alert) semantic web wherein constructs of tags of tags and winding hyperlinks reflecting states and relationships, not just for catching the great Diet Coke and Mentos research but to get the skinny on what the latest method is to build something, cure something, learn something, share something, price something, buy something, sell something.....
Or use the stored images to catch a thief (or worse) or provide compliance (not just for the SEC; as part of researching video applications for education, a fine gentleman named Mike Fink advised that some hospitals and emergency rooms are adopting video as anticipatory evidence when conducting particularly parlous procedures. (Perspicacity!)
The technology and methods approach not only Kurtzweil's "singularity" of intelligent machines, but the holodeck (hold on, Chief!) appears in the form of a walk about simulation of the hours before Pompeii's demise. The user of the system can move through that simulation by walking. (Heard that one on late night radio just before a discussion of alien abduction but I'm all about eclectic.) This, plus Wii systems have seen rapid adoption in physical therapy....
Consumers all over the joint.
Explicitly: this does not (yet) forecast the immediate demise of spinning media or tape, but the pricing pressures are and will continue to approach "unit costs of zero", with improving wide area connectivity which, while not allowing (for now) ignoring special relativity, multiple managed instances of information and stateful knowledge will appear more and more as abstractions within a deeply interconnected network. Seems not unreasonable to expect some form of bid/ask mechanisms to appear for dynamic allocation of, particularly, entertainment or thematically oriented information linking secure objects, generating additional copies, cleaning out the old data, and optimizing data (and processing) placement based upon dynamic needs.
Given that software itself decays, through maintenance in particular as cited in COCOMO models, etc., the management of storage more becomes the art of software as an increment to the art of understanding the more physical mechanisms of striping, RAID, densities, and mirrors. And the dark force of entropy, lo, will drive more preemptive testing and quality assurance (said the winner of sundry service packs).
And with the flat out impossibility of exhaustively testing every combination of lines of instructions (NP completeness means never having to say you're done) more nuanced and practical methods have emerged to do statistical measures akin to Design Of Experiment methods; however, still better hang on to that last working version.... Given that pretty much every buzzword required in a tech oriented article has been used here (oops, yeah, and Social Networking too!) we might as well toss in quantum computing.
And part of the quantum and string theory models is the many universes theory, which comforts me because it suggests, if true, I might finally be one of the smartest guys in the room, albeit a Cub Reporter.
Happy Spring!
Wednesday, March 12, 2008
What: availability. A how: Continuous Data Protection
Took a look at the published "inventory" of information on Google to give myself some orientation to the development timeline of what folks have been saying about availability (specifically"High Availability") and "Continuous Data Protection" to see when people started turning ideas into products.
The HA issues zipped right along from around 1985 or so (this is a survey, not a census, dear reader) with articulated specifications, formation of managed service offerings, products, etc. zipping right along to our current worlds.Continuous Data Protection, and by that, *that* particular search term shows up circa 1991 as prelim to disk mirroring products appearing later that decade.
The pre Sysplex days (and more people were working on the distributed problem than IBM) rested upon dark fiber, to me, reflecting the some people longing for dial tone at 40 pfennings a minute. SMDS, SONET offerings hadn't yet shown up, but the results were pretty convincing among some (rumored) blue sparks and flame that having trusted data in at least two places at once with a prayer (improvement) in recovering ones systems from the distributed data, well.... very good thing.
I'd argue, however, that the Continuous Data Protection model is the converged solution for how to answer the question of applications availability; the economics of (planned) redundancy favor that information distribution. Kindred concerns of custodial, compliance, and reliable connectivity, while significant, do invite innovations in putting the data objects. Market momentum for how to build higher availability into applications comes from known good libraries of "how to do this".
The DeDupe market space, as well, offers cost relief from the ability to recycle and realize more efficiencies in net storage capacities. The cautionary tale here comes from distributed computing, wherein some applications resemble Yorkie Terriers. Very very good at playing "this, all of this, is mine!" to the tune of "Big Boss Man" resulting with a conundrum of which manager manages the manager managers and a stack of dueling control systems oh heck lets put another piece of monitoring software in there that ought to hold 'em....
Which in turn brings back memories of notoriously brittle High Availability systems from the 90s, wherein the prudent management discipline was to establish that it was at last working and hanging up a sign that said "Was working fine when I left it. You broke it."
Some local use cases (involving moderate storage requirements and a thin network infrastructure) indicate that Continuous is the way to go (assuming that the data "containers or objects" allow for incremental updates). Saves network, and keeps one closer to the point in time when the fan will make a rude noise. Seriously looking at the peer to peer model has some wonderful attributes of survivability, and redundancy (boy, you can say that again) also with the potential for borrowing resources across networks.So in no way is it a motherhood issue as to how.
Barbie: Math is hard. She caught H E Double Hockeysticks for that but that's a fact.
Meanwhile, the what is of the motherhood issue (viz, a requirement to keep things going). But that how (one's chosen implementation). Hoo Wee! That how's a poser. But to me there's something in the thought that "this system swaps things about all of the time and keeps running with a provable audit trail of service levels" as more comforting than "it's in Lou's truck". One can always, as it were, burn a disk. Demonstrating recovery during system operation as a normal course of business.... cool.
The HA issues zipped right along from around 1985 or so (this is a survey, not a census, dear reader) with articulated specifications, formation of managed service offerings, products, etc. zipping right along to our current worlds.Continuous Data Protection, and by that, *that* particular search term shows up circa 1991 as prelim to disk mirroring products appearing later that decade.
The pre Sysplex days (and more people were working on the distributed problem than IBM) rested upon dark fiber, to me, reflecting the some people longing for dial tone at 40 pfennings a minute. SMDS, SONET offerings hadn't yet shown up, but the results were pretty convincing among some (rumored) blue sparks and flame that having trusted data in at least two places at once with a prayer (improvement) in recovering ones systems from the distributed data, well.... very good thing.
I'd argue, however, that the Continuous Data Protection model is the converged solution for how to answer the question of applications availability; the economics of (planned) redundancy favor that information distribution. Kindred concerns of custodial, compliance, and reliable connectivity, while significant, do invite innovations in putting the data objects. Market momentum for how to build higher availability into applications comes from known good libraries of "how to do this".
The DeDupe market space, as well, offers cost relief from the ability to recycle and realize more efficiencies in net storage capacities. The cautionary tale here comes from distributed computing, wherein some applications resemble Yorkie Terriers. Very very good at playing "this, all of this, is mine!" to the tune of "Big Boss Man" resulting with a conundrum of which manager manages the manager managers and a stack of dueling control systems oh heck lets put another piece of monitoring software in there that ought to hold 'em....
Which in turn brings back memories of notoriously brittle High Availability systems from the 90s, wherein the prudent management discipline was to establish that it was at last working and hanging up a sign that said "Was working fine when I left it. You broke it."
Some local use cases (involving moderate storage requirements and a thin network infrastructure) indicate that Continuous is the way to go (assuming that the data "containers or objects" allow for incremental updates). Saves network, and keeps one closer to the point in time when the fan will make a rude noise. Seriously looking at the peer to peer model has some wonderful attributes of survivability, and redundancy (boy, you can say that again) also with the potential for borrowing resources across networks.So in no way is it a motherhood issue as to how.
Barbie: Math is hard. She caught H E Double Hockeysticks for that but that's a fact.
Meanwhile, the what is of the motherhood issue (viz, a requirement to keep things going). But that how (one's chosen implementation). Hoo Wee! That how's a poser. But to me there's something in the thought that "this system swaps things about all of the time and keeps running with a provable audit trail of service levels" as more comforting than "it's in Lou's truck". One can always, as it were, burn a disk. Demonstrating recovery during system operation as a normal course of business.... cool.
"Say watt again. SAY WATT AGAIN"
Joules, old boy.
The conversations around the topic of "Green Computing" have focussed very much upon the management of heat loads in the metaphor of power. The technology itself heads toward a lower unit consumption of power production of heat due to smaller geometry as balanced against higher densities.
Once upon a time, R. Buckminster Fuller looked at the Spaceship Earth problems of electrical distribution and posited that the total capacities of power could on a global basis (more or less) be reduced (ceteris paribus) if the power grids of the whole world were deeply interconnected.
Sunny side of the planet, dark side of the planet, lighting from the sun, lighting from electricity, factories on, factories off. With some nit picking regarding transmission losses, etc. the proposition yields an interesting gedanken experiment vis a vis networks of computers and networks of storage. With some form of daily (whatever) variations, moving workloads, comprised of payloads of data and eventually application pieces, could let one reduce a given site's "peak capacity" through clever movement of processing.
Yes, latency. Yes, transmission capacities. Yes, etc.
But forms of this more agile distributive model come to us from, gasp, BitTorrent and other computing structures. For anyone who lives with real technology, the working model is that the solution will not be perfect (obsolete as it gets plugged in, in fact) but that the solution should be measurably "better".
We're living in a material world. You go girl.
Part of the Green computing ecosystem has been and will be the interaction of design for life cycles from a Cradle To Cradle point of view, increasing pressure on key materials in the computing industry (rare earths, gold), and improving recognition of the value of these materials in the cycle of creation, use, deconstruction, and re use. Fuller looked in particular at copper cycling through the system; the recycling of component materials in computing, however, has only recently become a design requirement. (LeRoy Budnik noted the "drive that will not die" in a recent post here.)
But the materials will be better "designed" to enable reuse in part because of "goodness" but principally because it makes sound organizational sense from the view of economics.
That the trend of ephemeralization (the less is more than more koan) cuts into this olio will be saved for another time.
When I have the erg.
With apologies.
The conversations around the topic of "Green Computing" have focussed very much upon the management of heat loads in the metaphor of power. The technology itself heads toward a lower unit consumption of power production of heat due to smaller geometry as balanced against higher densities.
Once upon a time, R. Buckminster Fuller looked at the Spaceship Earth problems of electrical distribution and posited that the total capacities of power could on a global basis (more or less) be reduced (ceteris paribus) if the power grids of the whole world were deeply interconnected.
Sunny side of the planet, dark side of the planet, lighting from the sun, lighting from electricity, factories on, factories off. With some nit picking regarding transmission losses, etc. the proposition yields an interesting gedanken experiment vis a vis networks of computers and networks of storage. With some form of daily (whatever) variations, moving workloads, comprised of payloads of data and eventually application pieces, could let one reduce a given site's "peak capacity" through clever movement of processing.
Yes, latency. Yes, transmission capacities. Yes, etc.
But forms of this more agile distributive model come to us from, gasp, BitTorrent and other computing structures. For anyone who lives with real technology, the working model is that the solution will not be perfect (obsolete as it gets plugged in, in fact) but that the solution should be measurably "better".
We're living in a material world. You go girl.
" Metals recirculate on a sum-total-of-all-metals-average every 22 1/2 years....
I was able to arrive at that figure of a 22 1/2-year metals recirculating cycle in 1936. I was working for Phelps Dodge Co., which had asked me to give them some prognostications about the uses of copper in the future of world industry."R. Buckminster Fuller, Critical Path
Part of the Green computing ecosystem has been and will be the interaction of design for life cycles from a Cradle To Cradle point of view, increasing pressure on key materials in the computing industry (rare earths, gold), and improving recognition of the value of these materials in the cycle of creation, use, deconstruction, and re use. Fuller looked in particular at copper cycling through the system; the recycling of component materials in computing, however, has only recently become a design requirement. (LeRoy Budnik noted the "drive that will not die" in a recent post here.)
But the materials will be better "designed" to enable reuse in part because of "goodness" but principally because it makes sound organizational sense from the view of economics.
That the trend of ephemeralization (the less is more than more koan) cuts into this olio will be saved for another time.
When I have the erg.
With apologies.
Tuesday, February 12, 2008
Bilbo! 'sup?
Just got power back here (ice storm, my woods went going off like rifle shots last night, my 125 pound dog skidded off front porch, and just got coffee at noon here).
Client to my south is MIA: the entire chunk of the state 10 miles south of me still goes without power. Could be a ton worse, I know.
It’s Tuesday, and I may have the truck chipped open by Thursday.
Regarding Virtual Tape I'm trying to find some kind of angle on it that hasn't been taken in before. Going back to some stuff I wrote earlier, I don't know that anyone would go into a tape environment if they didn't have existing backup software that expected to work with tape. Legacy apps hold back some of the transition.
The industry seems to consolidate towards Disk To Disk To Tape now; I'm trying to get to some true Disk Wallah friends to get better interpretation of that particular issue *but* seems that at least with that middle tier of disk I may be able to do more of the dedupe optimization because I don't have to muck with the serial access of the tape media.
I don't want to proclaim tape is dead, but it surely seems, from reading the "talking heads" usual suspects out there in the analyst community that it has really transformed: what matters is the virtual, not the tape. Still smelling intrinsic lock in, (so what, like, get a manager application to manage the manager manager application?) and there are many extant rants about the "who runs this" and unhappy back doors into the system to enable features like "if 2+2=5 then 2+2=5" kinds of kludges.
"One Ring to rule them all, One Ring to find them, One Ring to bring them all and in the darkness bind them" as Tolkien wrote.
I'm all over the caveats for the emptors, and suggest a vigorous scrubbing of any proposal to ensure that total costs and operational implications are walked through in detail. Something has to be "running" the infrastructure with assurance of the integrity of the data. De Dupe could be the planner who really doesn't game it out. I, truly, glaze over and turn to the serious people with scars and pocket protectors. My sense though is that tape fades: spinning moves to solid state, and even solid state becomes ephemeral with nano and finer grained media. The Virtual Tape business has saved millions of dollars (dozens of Loonies at today's rate) and solves a serious problem by abstracting the media, to a great extent, from the implementation. But a full model of "the business" can work wonders.
Think “The Sims” for Storage.
So maybe I should just post this? Another tree broke, dog is needy.
Client to my south is MIA: the entire chunk of the state 10 miles south of me still goes without power. Could be a ton worse, I know.
It’s Tuesday, and I may have the truck chipped open by Thursday.
Regarding Virtual Tape I'm trying to find some kind of angle on it that hasn't been taken in before. Going back to some stuff I wrote earlier, I don't know that anyone would go into a tape environment if they didn't have existing backup software that expected to work with tape. Legacy apps hold back some of the transition.
The industry seems to consolidate towards Disk To Disk To Tape now; I'm trying to get to some true Disk Wallah friends to get better interpretation of that particular issue *but* seems that at least with that middle tier of disk I may be able to do more of the dedupe optimization because I don't have to muck with the serial access of the tape media.
I don't want to proclaim tape is dead, but it surely seems, from reading the "talking heads" usual suspects out there in the analyst community that it has really transformed: what matters is the virtual, not the tape. Still smelling intrinsic lock in, (so what, like, get a manager application to manage the manager manager application?) and there are many extant rants about the "who runs this" and unhappy back doors into the system to enable features like "if 2+2=5 then 2+2=5" kinds of kludges.
"One Ring to rule them all, One Ring to find them, One Ring to bring them all and in the darkness bind them" as Tolkien wrote.
I'm all over the caveats for the emptors, and suggest a vigorous scrubbing of any proposal to ensure that total costs and operational implications are walked through in detail. Something has to be "running" the infrastructure with assurance of the integrity of the data. De Dupe could be the planner who really doesn't game it out. I, truly, glaze over and turn to the serious people with scars and pocket protectors. My sense though is that tape fades: spinning moves to solid state, and even solid state becomes ephemeral with nano and finer grained media. The Virtual Tape business has saved millions of dollars (dozens of Loonies at today's rate) and solves a serious problem by abstracting the media, to a great extent, from the implementation. But a full model of "the business" can work wonders.
Think “The Sims” for Storage.
So maybe I should just post this? Another tree broke, dog is needy.
Tuesday, January 15, 2008
Virtualization, Consumers, Networks, No Spin
A small company of 9 people called 37signals.com has a base of over 2,000,000 customers using one of their lead collaborative products. The company's excelled at "Goldilocks" design:not burdening software products with too much function.
From their blog at http://www.37signals.com/svn/posts/749-ask-37signals-numbers
Overall storage stats (Nov 2007)
They're going Xen, and reducing the number of servers by a little more than half.
What's more: they're a poster child for Amazon Simple Storage Service (Amazon S3).
Now, there's Google moving in this storage space, plus new players in the scrum, such as Nirvanix, and the Cleversafe, and who knows what that could be spun out from projects like (US) National LambdaRail.
Consumers
For a current project, looking to use "in the cloud" storage to host multimedia consumer "objects" and also musing upon the potential of little gadgets like iPod Touch to hold 10 hours of video with a price tag for the "1.0" of $300 US (which is, I think, about 80 Loonies this week). Since there's going to be a goat rodeo in producing content and control mechanisims for versioning and etc..... it becomes likely that we'll serve the content itself on an external network.
I'm lobbying for the Storage as a Super Hierarchy (wait for it: StaaSH) involving nothin' but net.
Life's too short.
Virtualization
Meanwhile, I keep looking at virtualization, and virtual tape. Spoke with a friend or two, and the interactivity of virtualization and de dupe starts to get interesting. I'm also of the opinion that no one would by choice run down the road of short tape for their application given the (wave hands) shifting cost curves of spinning drives, and especially considering the growth of solid state drives (I know, I know....) but in recognition also of nano tech showing up in what now..... 14 months.
So, my conjecture is that if one approached the problem set as a greenfield exercise, the pressure's very much in favor of non removable, and probably, non spinning media sooner rather than later. I'd be very interested in anyone's "fully burdened" cost analysis of this tech over time.
We used to have pizza boxes and pickles in the data center.....
No Spin
From their blog at http://www.37signals.com/svn/posts/749-ask-37signals-numbers
Overall storage stats (Nov 2007)
- 5.9 terabytes of customer-uploaded files
- 888 GB files uploaded (900,000 requests)
- 2 TB files downloaded (8,500,000 requests)
They're going Xen, and reducing the number of servers by a little more than half.
What's more: they're a poster child for Amazon Simple Storage Service (Amazon S3).
Now, there's Google moving in this storage space, plus new players in the scrum, such as Nirvanix, and the Cleversafe, and who knows what that could be spun out from projects like (US) National LambdaRail.
Consumers
For a current project, looking to use "in the cloud" storage to host multimedia consumer "objects" and also musing upon the potential of little gadgets like iPod Touch to hold 10 hours of video with a price tag for the "1.0" of $300 US (which is, I think, about 80 Loonies this week). Since there's going to be a goat rodeo in producing content and control mechanisims for versioning and etc..... it becomes likely that we'll serve the content itself on an external network.
I'm lobbying for the Storage as a Super Hierarchy (wait for it: StaaSH) involving nothin' but net.
Life's too short.
Virtualization
Meanwhile, I keep looking at virtualization, and virtual tape. Spoke with a friend or two, and the interactivity of virtualization and de dupe starts to get interesting. I'm also of the opinion that no one would by choice run down the road of short tape for their application given the (wave hands) shifting cost curves of spinning drives, and especially considering the growth of solid state drives (I know, I know....) but in recognition also of nano tech showing up in what now..... 14 months.
So, my conjecture is that if one approached the problem set as a greenfield exercise, the pressure's very much in favor of non removable, and probably, non spinning media sooner rather than later. I'd be very interested in anyone's "fully burdened" cost analysis of this tech over time.
We used to have pizza boxes and pickles in the data center.....
No Spin
Tuesday, December 11, 2007
Black Friday Haiku.... New Storage Demands
1Gig MP3 Player At Discount Chain $30
20Gb Local ISP "data backup" $ 600 USD/Year
40 Gb Google extra storage $75 USD/Year
Not a very large sample, but thecreation and consumerization of the datasphere's happening apace. Time to dust off the "evergreen" plan and consider the innovation rate plus (for the non rural consumer) improving network speeds.
This may be old news but... several companies (my own employer included) in the days after 9/11 successfully restored the formal data and resumed operations.
The informal data, PDAs, rollodex, abandoned or destroyed laptops, etc. held the heart and soul of relationship information comprising customer contact info and the aggregation of information, most of it informal.
BOHICA, folks.
The complexity and pure units x density calculations will be hitting infrastructures across industries.And from the (rigorous!) pricing example above, the growth of ubiquitous personal and informal data, showing up on iPods, digital photo frames, compliance logs of videoconferences or morning calls, security systems, entertainment....
Well, disruptive would be a mild term.
20Gb Local ISP "data backup" $ 600 USD/Year
40 Gb Google extra storage $75 USD/Year
Not a very large sample, but thecreation and consumerization of the datasphere's happening apace. Time to dust off the "evergreen" plan and consider the innovation rate plus (for the non rural consumer) improving network speeds.
This may be old news but... several companies (my own employer included) in the days after 9/11 successfully restored the formal data and resumed operations.
The informal data, PDAs, rollodex, abandoned or destroyed laptops, etc. held the heart and soul of relationship information comprising customer contact info and the aggregation of information, most of it informal.
BOHICA, folks.
The complexity and pure units x density calculations will be hitting infrastructures across industries.And from the (rigorous!) pricing example above, the growth of ubiquitous personal and informal data, showing up on iPods, digital photo frames, compliance logs of videoconferences or morning calls, security systems, entertainment....
Well, disruptive would be a mild term.
Tuesday, October 30, 2007
You give me fervor: dSt/dt = a(C, P, ...) [St/Ut] * [1 - St/Ut]
Having made stuff up conducted trend analysis and flipped coins integrated qualitative research for about 30 years, this generation of nets and boxes begins to rock in pretty disruptive ways. In this new place, massive networks of storage, more consumer driven than ever, connected by faster networks and wicked magic small tech.We've only hadaspirin around for 110 years. Modeling complex systems with itty bitty Reverse Polish Notation in the hallowed HP-45's ten or so registers and 49 programming steps.Writing decision support programs in Lotus 1-2-3 with 5 inch floppies.... and that dreamy IBM PC XT with what, 10 or 20 droolworthy Megabytes of storage. Then HyperCard and hmmm.... I think it was Mac with 6.something used to model Automatic Teller Machine user interfaces with real live consumers pointing and clicking.... And these kids today: so many tubes!Adoption curves (usually s shaped, often tied with knowing marketing hosers arguing either for or against the adoption of the Next Big Thing depending upon whether said hoser has (or can assert ownership of) the Next Big Thing) apply, but do the curves start to point ever more upward?More people. More networks. More transparency to the tech. Better abstraction and generalization of interfaces (none dare call it commoditization). Like whoosh? Potentiated equilibrium? Chicxulub?
Andy S. Kydes for the US Department of Energy provides a concise review of how capital budgeting decisions occur in the context of changing technology. (The model best applies, I believe, to infrastructure decisions which tend to be big lumps of capital and concurrent retooling of skills, but is nonetheless useful in exposition of the dynamics of market adoption.)The "dSt/dt = a(C, P, ...) [St/Ut] * [1 - St/Ut]" bits concern, essentially, the rate at which an infection moves through a population. Technology adoption follows this kind of logic. A few try it, they like it, they tell their community and the new drives out (or back) most (sometimes all) of the old.NB: Experienced technology managers will recognize that for early releases the infection model truly speaks truth. Although the DOE's exposition of tech adoption does involve energy components (coal, nukes, etc.) but the principles of how one generally assesses and then opts to adopt a new technology inform the supply side of storage componentry and the demand side of direct storage consumers (thumbdrives) and service providers employing storage as a means to an end (Google, Carbonite, HuLu, yadda yadda).What continues apace: Acceleration Interesting to see the delivery of high capacity spinning media, high density flash media (Gb), and emergent nanotech promising huge efficiencies and availability in something like 18 months (mid-2009) in terabytes.Well, interesting in the sense of wonks, not interesting like "the promise of moonlight in a martini", as Mr. Shanley noted.Curious as to how the Technology Adoption Life Cycle morphs. Do we all become Early Adopters? Seems that time to market becomes the critical success factor.... I mean.... like really really vital on the supply side (sell 'em) and the demand side (buy 'em for the benefits). Hmm.... so let's say I can abstract the command and control systems in some stable overlay that permits rapid change out in underlying components, thereby reducing the friction of changing out the So Last Year blinkenlights for the So Happening new blinkenlights. Doodling time. Time for coffee. Or moonlight.
Andy S. Kydes for the US Department of Energy provides a concise review of how capital budgeting decisions occur in the context of changing technology. (The model best applies, I believe, to infrastructure decisions which tend to be big lumps of capital and concurrent retooling of skills, but is nonetheless useful in exposition of the dynamics of market adoption.)The "dSt/dt = a(C, P, ...) [St/Ut] * [1 - St/Ut]" bits concern, essentially, the rate at which an infection moves through a population. Technology adoption follows this kind of logic. A few try it, they like it, they tell their community and the new drives out (or back) most (sometimes all) of the old.NB: Experienced technology managers will recognize that for early releases the infection model truly speaks truth. Although the DOE's exposition of tech adoption does involve energy components (coal, nukes, etc.) but the principles of how one generally assesses and then opts to adopt a new technology inform the supply side of storage componentry and the demand side of direct storage consumers (thumbdrives) and service providers employing storage as a means to an end (Google, Carbonite, HuLu, yadda yadda).What continues apace: Acceleration Interesting to see the delivery of high capacity spinning media, high density flash media (Gb), and emergent nanotech promising huge efficiencies and availability in something like 18 months (mid-2009) in terabytes.Well, interesting in the sense of wonks, not interesting like "the promise of moonlight in a martini", as Mr. Shanley noted.Curious as to how the Technology Adoption Life Cycle morphs. Do we all become Early Adopters? Seems that time to market becomes the critical success factor.... I mean.... like really really vital on the supply side (sell 'em) and the demand side (buy 'em for the benefits). Hmm.... so let's say I can abstract the command and control systems in some stable overlay that permits rapid change out in underlying components, thereby reducing the friction of changing out the So Last Year blinkenlights for the So Happening new blinkenlights. Doodling time. Time for coffee. Or moonlight.
Subscribe to:
Posts
(
Atom
)