As I frequently remind my children:
It is better to be happy about the things you have,
than unhappy about the things you lack.
As I frequently remind my children:
It is better to be happy about the things you have,
than unhappy about the things you lack.
On May 9th, 2013 it will be the 10th anniversary of the issuance of the DoD’s Net-Centric Data Strategy. In honor of this, I would like to highlight a collection of slides that were used (many, many times) to illustrate the concepts of net-centricity, the data strategy, Communities of Interest, and User-Defined Operational Pictures.
These slides are quite dated… last updated in 2006, but there are some great illustrations and concepts that remain relevant. The Data Strategy was a brilliant work that had great promise for changing the way people see IT in defense, in my opinion. It’s tragic that various personnel changes and other events shifted the focus elsewhere.
[Extra disclaimer: I am not currently involved in any deliberations with regard to AHLTA or VistA or iEHR, and have no particular knowledge of current thinking on EHR inside the Pentagon. I briefed the Hon. Beth McGrath on Open Source Software in April 2011 – a long time ago.]
As some of you probably know, Jon Stewart of the Daily Show recently ran a clip about AHLTA vs VistA. (embedded below for convenience)
One of the interesting side-stories from this is the open-source vs proprietary angle which Stewart did not address. The VA was (is?) a big supporter of Open Source Software. (It is less clear how much commitment remains, now that both Roger Baker and Peter Levin have left the VA.)
An Open Source Software model makes perfect sense for VA. They had the business problem of 133 treatment facilities with 133 custom versions of VistA. One of the beautiful things about VistA was that it was open to customization at the various installations so that it could be tailored to specific needs and this allowed innovation at point of service. The problem is that the overall operational cost increases over time as the versions diverge, and there is the possibility that interoperability decreases. The VA created OSEHRA to bring these forks back together. By using open source methodologies, they created a common “VA Enterprise VistA” baseline, gives a place for all those local improvements to be merged into a best-of-breed VistA.
This collaborative model is a huge win for innovation and continuous modernization. Allowing local innovation is both good and bad: the innovation part is good, but the loss of standardization is bad. Coupling a tolerance for local innovation with a enterprise reference standard makes the prospect win-win.
I’ve personally fixed three bugs in the Linux kernel, and successfully gotten those patches incorporated into the Torvalds-approved kernel.org kernel. I fixed the bugs because they affected me personally. I worked to get them into the upstream because I didn’t want to have to keep re-applying my patches every time a new kernel was released. There is also a non-trivial pride and ego-boost associated with that accomplishment. These same incentives will cause VA doctors and IT staff to work to improve VistA locally, and also to merge those changes into an enterprise baseline.
These arguments in favor of an open-source development model could apply to the DoD also, or to a joint DoD-VA approach.
What’s not clear to me is how much VistA should play a starring role in that future.
Here’s an issue: building a vibrant, collaborative community around a software development project is about people and process as much as it is about technology. Some great books have been written about this. It will be difficult to get bright, talented software engineers to work on VistA, because of the 30-year old tools and design practices. I have two degrees in computer science from MIT and you couldn’t pay me enough to work on VistA. Literally. Even if you aren’t a software engineer, take a look at this random sample of VistA source code, which is barely distinguishable from line noise:
Not all the VistA code is this bad; some of it is worse.
That said, the DoD alternative (AHLTA) – and in fact the most well-known proprietary alternative – all share the same obscure programming language (MUMPS) and are almost certainly equally bad – but since their code is not public, it’s harder to critique them. In fact, I would expect them to be worse since they were not written with collaborative development in mind.
Say all the nice things about MUMPS you want: In the end, the choice of an ugly, archaic technology will decrease interest in any project by prospective contributors, thus decreasing the value of the collaborative model. This is Technical Debt we may not be able to repay.
Interestingly, Philip Newcomb, CEO of The Software Revolution Inc., has asserted that his company’s technology could convert VistA from MUMPS to J2EE in about a year for $10m. If true, this would be a bargain, IMHO. I’ve met with Phil twice in the past decade, and his claims are impressive, although I have no personal experience with the results of his company’s work. Other companies perform similar services – Hatha Systems for example claims to do automated analysis of MUMPS.
Tom Munnecke, one of the original architects of VistA has eloquently defended the MUMPS database design, pointing out that medical data is rarely suited to the structured, SQL-esque approach of relational databases. He makes some fabulous points, and I actually think that he’s right in that many of the decisions that were made for MUMPS and VistA were remarkably prescient. It’s taken the rest of the IT world three decades to rediscover document-structured non-relational databases, now known collectively as NoSQL. Now that there’s a huge-amount of energy and expertise focused on large-scale non-relational data stores, maybe we should consider how to use that talent and energy for EHR?
In short, I think the VA should keep doing the OSEHRA thing to consolidate and modernize VistA. There’s two threads to that: first, they need to consolidate into an enterprise version of VistA for the VA to bring together the forks (which they are doing), and second, they should refactor and modernize the architecture and the tooling (which they claim they are doing). I am suspicious that they aren’t being bold enough, but I don’t know.
I remain skeptical that VistA can survive in the long term as a vibrant, community-driven open source project, if it continues as it is. In order to make VistA a viable project, the current MUMPS-based database need to be replaced with a modern NoSQL datastore of some kind, and the hyper-abbreviated MUMPS code needs to be replaced with something readable and maintainable. A colleague of mine (David Wheeler) once pointed out that MUMPS code doesn’t have to be unreadable, but the coding practices of VistA do not lend themselves to readability. A bold step would be to re-write the thing in Java or some other modern language; a mimimalist step would just re-write it in MUMPS that doesn’t suck so much. In David’s words:
I think you should note another alternative as well: Keep MUMPS, but translate the current MUMPS into readable code.
Yes, you’d still be working in an uncommon language. But that transformation would be especially trivial to do (and trivial to automate), and the risks from auto-translation would be far lower because the underlying environment and assumptions would be unchanged.
It seems to me that the big problem here isn’t really MUMPS, it’s the way MUMPS has been used. Developers have used MUMPS’ “you can abbreviate anything” combined with “use bad names”, which perhaps made sense 30 years ago but is a bad idea today. But you do not *HAVE* to create ugly code in MUMPS.
Using the Wikipedia MUMPS article example, here’s some line noise:hello() w "Hello, World!",! q
But here’s legal MUMPS – it’s the same code, but unencrypted:hello() write "Hello, World!",! quit
I have no idea if translating to another language would be a better trade-off than translating it to readable MUMPS. But it’d be easy to hire somebody to briefly investigate the options, pros, and cons, so that a reasonable decision (based on EVIDENCE) could be made. And I think, in fairness, that alternative should be considered.
I also have concerns about the governance model of OSEHRA, but, as this blog post is already too long, I’ll save that for another article.
As promised at the top: Jon Stewart on AHLTA vs VistA:
I have a guilty and shameful confession to make.
It happened years ago, and it’s been troubling me ever since. I didn’t understand my mistake until recently. It’s a little hard to explain, so bear with me.
Sometime around 2007, I happened to be at the old DISA Skyline-7 building, when a project called “Button Two” was unveiled. Continue reading
[Update, 1 May 2014: The issue of 1st amendment free-speech protection as a fed is more complicated than I knew at the time this post was written. I’ve recently learned of the 2006 Supreme Court Decision, Garcetti v. Ceballos, which significantly restricts the protection for speech as a government employee. I will probably write a new post once I’ve read the Court’s opinions and understood the implications thereof.]
I got in trouble for blogging. Sorta. And I got applauded too. More on that later.
One of the posts on my blog got noticed. People were offended that I would speak out of turn. Apparently, a senior military officer printed a copy of the post and handed it to my boss’s boss’s boss, and recommended that I receive ethics counseling. I suspect that he has not read the Cluetrain Manifesto.
To the great credit of my leadership, they took this under advisement. Some of them even read the blog. One of the comments that got back to me was, “That was a well written post.” My supervisor did call the Standards of Conduct Office, and he recommended that I go speak with an Ethics Official (an attorney), just so I would know what the boundaries were, and what I could (or could not) do. I met with a very informed and articulate attorney and had a pleasant conversation with her.
Here is a synopsis of what I’ve learned about defense-civilian-blogger’s rights: (side commentary in green)
That was my official “ethics counseling” on blogging as a federal civilian. There was one additional issue, concerning cross-posting to Intelink-U. Separately, I asked this question to a colleague on Intelink staff, who passed it on to local counsel. Here is an excerpt from the response, which is too precious not to share:
Having read the blog, I have couple of observations on the blog and its content–
- A blog, by definition, is personal commentary on a relevant topic.
- I see nothing in the content that conflicts with the Federal employee ethics regulations codified in 5 C.F.R. Part 2635: Standards of ethical conduct for employees of the executive branch.
Based on a close read of Mr. Risacher comments, it could be reasonably argued that his commentary directly relates to his ethical obligations under 5 CFR Part 2635.101 (b) (11), “Employees shall disclose waste, fraud, abuse, and corruption to appropriate authorities. Albeit, it is not “politically correct/wise” to use a public forum as the method of “disclosure.”
His use of a “real” example to make his points about the well known disconnects between many “program management” and “operations” activities. Without such a concrete example, however, his commentary would have been vague and hypothetical, and dismissed as such.
And, oh by the way, while he is not a DNI employee one of the core values and performance measure is Courage…”moral, intellectual, even physical. We have the integrity (indeed, the duty) to seek and speak the truth, to innovate, to change things for the better, regardless of personal or professional risk.” In fact, a key assessment criteria for all DNI employees under the Personal Leadership and Integrity section is “a commitment to excellence, and the courage and conviction to express their professional views”
There are a few relevant questions one should consider:
Is is therefore appropriate to punish Mr. Risacher for doing what would be demanded of an IC professional?
In fact, is it not contrary to such principles to suggest such a notion?
He has effectively thrown a “stupid flag” and now someone is more concerned about the fact that his comments might be professionally embarrassing/inconvenient. Nowhere in the ethics regulations, does it say that it is ethical to shoot the messenger of inconvenient truths.
Amen. Thank you.
I’ve been reading the Cluetrain Manifesto (10th Anniversary Edition), a book that examines the impact of the Internet on both markets (consumers) and organizations. The over-the-top introduction reads like this (with minor editorial
A powerful global conversation has begun. Through the Internet, people are discovering and inventing new ways to share relevant knowledge with blinding speed. As a direct result, markets are getting smarter—and getting smarter faster than most
These markets are conversations. Their members communicate in language that is natural, open, honest, direct, funny and often shocking. Whether explaining or complaining, joking or serious, the human voice is unmistakably genuine. It can’t be faked. Continue reading