[Extra disclaimer: I am not currently involved in any deliberations with regard to AHLTA or VistA or iEHR, and have no particular knowledge of current thinking on EHR inside the Pentagon. I briefed the Hon. Beth McGrath on Open Source Software in April 2011 – a long time ago.]
As some of you probably know, Jon Stewart of the Daily Show recently ran a clip about AHLTA vs VistA. (embedded below for convenience)
One of the interesting side-stories from this is the open-source vs proprietary angle which Stewart did not address. The VA was (is?) a big supporter of Open Source Software. (It is less clear how much commitment remains, now that both Roger Baker and Peter Levin have left the VA.)
An Open Source Software model makes perfect sense for VA. They had the business problem of 133 treatment facilities with 133 custom versions of VistA. One of the beautiful things about VistA was that it was open to customization at the various installations so that it could be tailored to specific needs and this allowed innovation at point of service. The problem is that the overall operational cost increases over time as the versions diverge, and there is the possibility that interoperability decreases. The VA created OSEHRA to bring these forks back together. By using open source methodologies, they created a common “VA Enterprise VistA” baseline, gives a place for all those local improvements to be merged into a best-of-breed VistA.
This collaborative model is a huge win for innovation and continuous modernization. Allowing local innovation is both good and bad: the innovation part is good, but the loss of standardization is bad. Coupling a tolerance for local innovation with a enterprise reference standard makes the prospect win-win.
I’ve personally fixed three bugs in the Linux kernel, and successfully gotten those patches incorporated into the Torvalds-approved kernel.org kernel. I fixed the bugs because they affected me personally. I worked to get them into the upstream because I didn’t want to have to keep re-applying my patches every time a new kernel was released. There is also a non-trivial pride and ego-boost associated with that accomplishment. These same incentives will cause VA doctors and IT staff to work to improve VistA locally, and also to merge those changes into an enterprise baseline.
These arguments in favor of an open-source development model could apply to the DoD also, or to a joint DoD-VA approach.
What’s not clear to me is how much VistA should play a starring role in that future.
Here’s an issue: building a vibrant, collaborative community around a software development project is about people and process as much as it is about technology. Some great books have been written about this. It will be difficult to get bright, talented software engineers to work on VistA, because of the 30-year old tools and design practices. I have two degrees in computer science from MIT and you couldn’t pay me enough to work on VistA. Literally. Even if you aren’t a software engineer, take a look at this random sample of VistA source code, which is barely distinguishable from line noise:
GMRCEDT1 ;SLC/DCM,JFR - EDIT A CONSULT AND RE-SEND AS NEW ;08/20/09 12:16
;;3.0;CONSULT/REQUEST TRACKING;**1,5,12,15,22,33,47,66**;DEC 27, 1997;Build 30
;
; This routine invokes IA #2638 (^ORD(100.01,), #3991 (ICDAPIU), #10117 (VALM10)
; #10103 (XLFDT), #10104 (XLFSTR), #10060 (access ^VA(200)), #2056 (GET1^DIQ)
;
EN(GMRCO) ;GMRCO=IEN of consult from file 123
;GMRCSS=To Service GMRCPROC=Procedure Request Type
;GMRCURG=Urgency GMRCPL=Place Of Consultation
;GMRCATN=Attention GMRCINO=Service is In/Out Patient
;GMRCPNM=Patient Name GMRCDIAG=Provisional Diagnosis
;GMRCERDT=Earliest Appr. Date
;GMRCPROS=Prosthetics Service y/n
N GMRCSS,GMRCPROC,GMRCURG,GMRCPL,GMRCATN,GMRCINO,GMRCDIAG,LN,GMRCRESP,GMRCERDT,GMRCPROS,IDX
K ^TMP("GMRCR",$J,"ED") S GMRCLNO=1
I $L($P(^GMR(123,+GMRCO,0),"^",12)) S ^TMP("GMRCR",$J,"ED",GMRCLNO,0)=" CURRENT STATUS: (Not Editable): "_$P(^ORD(100.01,$P(^(0),"^",12),0),"^",1),GMRCLNO=GMRCLNO+1
S GMRCD=0 F S GMRCD=$O(^GMR(123,+GMRCO,40,"B",GMRCD)) Q:'GMRCD S GMRCDD="" F S GMRCDD=$O(^GMR(123,GMRCO,40,"B",GMRCD,GMRCDD)) Q:'GMRCDD D
.I $P(^GMR(123,+GMRCO,40,GMRCDD,0),"^",2)=19 S LN=0 D
..N GMRCPERS,GMRCTX
..I '$D(^GMR(123,+GMRCO,12)) D
...S GMRCPERS=+$P($G(^GMR(123,+GMRCO,40,GMRCDD,0)),"^",5)
...S GMRCPERS=$GET1^DIQ(200,GMRCPERS,.01)
..I $D(^GMR(123,+GMRCO,12)) D
...I $P(^GMR(123,+GMRCO,12),U,5)="P" D
....S GMRCPERS=$P($G(^GMR(123,+GMRCO,40,GMRCDD,2)),U,1)
...I $P(^GMR(123,+GMRCO,12),U,5)="F" D
....S GMRCPERS=$P($G(^GMR(123,+GMRCO,40,GMRCDD,0)),U,5)
....S GMRCPERS=$GET1^DIQ(200,GMRCPERS,.01)
..S GMRCTX=" CANCELLED BY (Not Editable): "_GMRCPERS
..S ^TMP("GMRCR",$J,"ED",GMRCLNO,0)=GMRCTX,GMRCLNO=GMRCLNO+1
..S ^TMP("GMRCR",$J,"ED",GMRCLNO,0)=" CANCELLED COMMENT (Not Editable):",GMRCLNO=GMRCLNO+1
..S LN=$O(^GMR(123,+GMRCO,40,GMRCDD,1,LN)) Q:LN=""!(LN?1A.E) I $L(^GMR(123,+GMRCO,40,GMRCDD,1,LN,0))>75 S FLG=1 D WPSET^GMRCUTIL("^GMR(123,+GMRCO,40,GMRCDD,1)","^TMP(""GMRCR"",$J,""ED"")","",.GMRCLNO,"",FLG)
..I '$D(FLG) S LN=0 F S LN=$O(^GMR(123,+GMRCO,40,GMRCDD,1,LN)) Q:LN=""!(LN?1A.E) S ^TMP("GMRCR",$J,"ED",GMRCLNO,0)=^(LN,0),GMRCLNO=GMRCLNO+1
..S ^TMP("GMRCR",$J,"ED",GMRCLNO,0)="",$P(^(0),"-",79)=""
..S GMRCLNO=GMRCLNO+1
..Q
.Q
S GMRCSS=$S($D(GMRCEDT(1)):GMRCEDT(1),1:$P(^GMR(123,+GMRCO,0),"^",5)_U_$P(^GMR(123.5,$P(^GMR(123,+GMRCO,0),"^",5),0),U))
S GMRCPROC=$S($D(GMRCED(1)):GMRCED(1),1:$P(^GMR(123,+GMRCO,0),"^",8)_U_$GET1^DIQ(123.3,+$P(^GMR(123,+GMRCO,0),"^",8),.01))
I $D(GMRCED(2)) S GMRCINO=GMRCED(2)
I '$D(GMRCINO) S GMRCINO=$P(^GMR(123,+GMRCO,0),U,18)_U_$S($P(^(0),U,18)="I":"Inpatient",1:"Outpatient")
S GMRCURG=$S($D(GMRCED(3)):GMRCED(3),1:$P(^GMR(123,+GMRCO,0),"^",9)_U_$GET1^DIQ(101,+$P(^(0),"^",9),1))
S GMRCPL=$S($D(GMRCED(4)):GMRCED(4),1:$P(^GMR(123,+GMRCO,0),"^",10)_U_$GET1^DIQ(101,+$P(^(0),U,10),1))
S GMRCPROS=$G(^GMR(123.5,$P(GMRCSS,U),"INT")) I $G(GMRCPROS)="" D
.S GMRCERDT=$S($D(GMRCED(5)):GMRCED(5),1:$P(^GMR(123,+GMRCO,0),"^",24)_U_$FMTE^XLFDT($P(^GMR(123,GMRCO,0),U,24)))
S GMRCATN=$S($D(GMRCED(6)):GMRCED(6),1:$P(^GMR(123,+GMRCO,0),"^",11)_U_$GET1^DIQ(200,+$P(^(0),U,11),.01))
I '$D(^GMR(123,GMRCO,30.1)) D
. I $D(GMRCED(7)),$L($P(GMRCED(7),U,2)) D Q
.. S GMRCDIAG=$P(GMRCED(7),U)_" ("_$P(GMRCED(7),U,2)_")"
. S GMRCDIAG=$S($D(GMRCED(7)):GMRCED(7),1:$G(^GMR(123,+GMRCO,30)))
I $D(^GMR(123,GMRCO,30.1)) D
. I $D(GMRCED(7)),$L(GMRCED(7)) D Q
.. S GMRCDIAG=$P(GMRCED(7),U)_" ("_$P(GMRCED(7),U,2)_")"
. S GMRCDIAG=$G(^GMR(123,+GMRCO,30))
. I '$STATCHK^ICDAPIU(^GMR(123,GMRCO,30.1),DT) D
.. S GMRCDIAG=GMRCDIAG_" "
S GMRCREQ=$S(+$P(^GMR(123,+GMRCO,0),U,17)="P":"Procedure",1:"Consult")
S ^TMP("GMRCR",$J,"ED",GMRCLNO,0)="SENDING PROVIDER (Not Editable): "_$S($P($G(^GMR(123,+GMRCO,12)),U,6):$P(^GMR(123,+GMRCO,12),U,6),$P(^GMR(123,+GMRCO,0),"^",14):$GET1^DIQ(200,+$P(^GMR(123,+GMRCO,0),"^",14),.01),1:"UNKNOWN"),GMRCLNO=GMRCLNO+1
S ^TMP("GMRCR",$J,"ED",GMRCLNO,0)="REQUEST TYPE (Not Editable): "_GMRCREQ,GMRCLNO=GMRCLNO+1
S ^TMP("GMRCR",$J,"ED",GMRCLNO,0)=$REPEAT^XLFSTR("-",79),GMRCLNO=GMRCLNO+1
S ^TMP("GMRCR",$J,"ED",GMRCLNO,0)=" TO SERVICE (Not Editable): "_$P(GMRCSS,U,2) S GMRCLNO=GMRCLNO+1
S ^TMP("GMRCR",$J,"ED",GMRCLNO,0)=" ",GMRCLNO=GMRCLNO+1
S IDX=1,^TMP("GMRCR",$J,"ED",GMRCLNO,0)=IDX_" PROCEDURE: "_$P(GMRCPROC,U,2)
D:+GMRCPROC RVRS(GMRCLNO,$D(GMRCED(IDX))) S GMRCLNO=GMRCLNO+1,IDX=IDX+1
S ^TMP("GMRCR",$J,"ED",GMRCLNO,0)=IDX_" Performed as INPT OR OUTPT: "_$P(GMRCINO,U,2) D RVRS(GMRCLNO,$D(GMRCED(2))) S GMRCLNO=GMRCLNO+1,IDX=IDX+1
S ^TMP("GMRCR",$J,"ED",GMRCLNO,0)=IDX_" URGENCY: "_$P(GMRCURG,U,2) D RVRS(GMRCLNO,$D(GMRCED(3))) S GMRCLNO=GMRCLNO+1,IDX=IDX+1
S ^TMP("GMRCR",$J,"ED",GMRCLNO,0)=IDX_" PLACE OF CONSULTATION: "_$P(GMRCPL,U,2) D RVRS(GMRCLNO,$D(GMRCED(4))) S GMRCLNO=GMRCLNO+1,IDX=IDX+1
I $G(GMRCPROS)="" S ^TMP("GMRCR",$J,"ED",GMRCLNO,0)=IDX_" EARLIEST APPROPRIATE DATE: "_$P(GMRCERDT,U,2) D RVRS(GMRCLNO,$D(GMRCED(5))) S GMRCLNO=GMRCLNO+1,IDX=IDX+1
S ^TMP("GMRCR",$J,"ED",GMRCLNO,0)=IDX_" ATTENTION (CONSULTANT): "_$P(GMRCATN,U,2) D RVRS(GMRCLNO,$D(GMRCED(6))) S GMRCLNO=GMRCLNO+1,IDX=IDX+1
S ^TMP("GMRCR",$J,"ED",GMRCLNO,0)=IDX_" PROVISIONAL DIAGNOSIS: "_GMRCDIAG D RVRS(GMRCLNO,$D(GMRCED(7))) S GMRCLNO=GMRCLNO+1,IDX=IDX+1
S ^TMP("GMRCR",$J,"ED",GMRCLNO,0)=IDX_" REASON FOR REQUEST:" D RVRS(GMRCLNO,$D(^TMP("GMRCED",$J,20))) S GMRCLNO=GMRCLNO+1,IDX=IDX+1 D
. I $D(^TMP("GMRCED",$J,20)) D Q
.. N ND S ND=0
.. F S ND=$O(^TMP("GMRCED",$J,20,ND)) Q:'ND D
... D KILL^VALM10(GMRCLNO)
... S ^TMP("GMRCR",$J,"ED",GMRCLNO,0)=^TMP("GMRCED",$J,20,ND,0)
... S GMRCLNO=GMRCLNO+1
. N ND S ND=0
. F S ND=$O(^GMR(123,+GMRCO,20,ND)) Q:ND="" D
.. S ^TMP("GMRCR",$J,"ED",GMRCLNO,0)=^GMR(123,+GMRCO,20,ND,0)
.. S GMRCLNO=GMRCLNO+1
.Q
S ^TMP("GMRCR",$J,"ED",GMRCLNO,0)="",GMRCLNO=GMRCLNO+1,^TMP("GMRCR",$J,"ED",GMRCLNO,0)=IDX_" COMMENT(S): (Add Only)" D RVRS(GMRCLNO) S GMRCLNO=GMRCLNO+1
I $D(^TMP("GMRCED",$J,40)) D
. D KILL^VALM10(GMRCLNO)
. S ^TMP("GMRCR",$J,"ED",GMRCLNO,0)=" New Comment:",GMRCLNO=GMRCLNO+1
. N ND S ND=0 F S ND=$O(^TMP("GMRCED",$J,40,ND)) Q:'ND D
.. S ^TMP("GMRCR",$J,"ED",GMRCLNO,0)=^TMP("GMRCED",$J,40,ND,0)
.. S GMRCLNO=GMRCLNO+1
N GMRCEDCT
S GMRCD=0,GMRCEDCT=0 F S GMRCD=$O(^GMR(123,+GMRCO,40,"B",GMRCD)) Q:'GMRCD S GMRCDD="",GMRCDD=$O(^GMR(123,+GMRCO,40,"B",GMRCD,GMRCDD)) Q:'GMRCDD D
.I $P(^GMR(123,+GMRCO,40,GMRCDD,0),"^",2)=20 S LN=0,GMRCEDCT=GMRCEDCT+1,GMRCEDCM(GMRCEDCT)=GMRCDD D
..S ^TMP("GMRCR",$J,"ED",GMRCLNO,0)="",GMRCLNO=GMRCLNO+1,^TMP("GMRCR",$J,"ED",GMRCLNO,0)="ADDED COMMENT (Not Editable) Entered: "_$P($FMTE^XLFDT($P(^GMR(123,+GMRCO,40,GMRCDD,0),"^",1)),"@",1)
..S GMRCRESP=$S($L($P($G(^GMR(123,+GMRCO,40,GMRCDD,0)),"^",5)):$GET1^DIQ(200,$P(^GMR(123,+GMRCO,40,GMRCDD,0),"^",5),.01),$L($P($G(^GMR(123,+GMRCO,40,GMRCDD,2)),"^",1)):$P(^GMR(123,+GMRCO,40,GMRCDD,2),"^",1),1:"UNKNOWN")
..S ^TMP("GMRCR",$J,"ED",GMRCLNO,0)=^TMP("GMRCR",$J,"ED",GMRCLNO,0)_" BY: "_GMRCRESP,GMRCLNO=GMRCLNO+1
..;S ^TMP("GMRCR",$J,"ED",GMRCLNO,0)=^TMP("GMRCR",$J,"ED",GMRCLNO,0)_" BY: "_$S($L($P(^GMR(123,+GMRCO,40,GMRCDD,0),"^",4)):$P(^VA(200,$P(^GMR(123,+GMRCO,40,GMRCDD,0),"^",4),0),"^",1),1:"UNKNOWN"),GMRCLNO=GMRCLNO+1
..S LN=$O(^GMR(123,+GMRCO,40,GMRCDD,1,LN)) Q:LN=""!(LN?1A.E) I $L(^GMR(123,+GMRCO,40,GMRCDD,1,LN,0))>75 S FLG=1 D WPSET^GMRCUTIL("^GMR(123,+GMRCO,40,GMRCDD,1)","^TMP(""GMRCR"",$J,""ED"")","",.GMRCLNO,"",FLG) Q
..S LN=0 F S LN=$O(^GMR(123,+GMRCO,40,GMRCDD,1,LN)) Q:LN=""!(LN?1A.E) S ^TMP("GMRCR",$J,"ED",GMRCLNO,0)=^(LN,0),GMRCLNO=GMRCLNO+1
..Q
.Q
S ^TMP("GMRCR",$J,"ED",GMRCLNO,0)=""
K FLG
Q
RVRS(LINE,EDITED) ;reverse video for fields that can be edited
I '$G(EDITED) D CNTRL^VALM10(LINE,1,1,IORVON,IORVOFF) Q
D CNTRL^VALM10(LINE,1,1,IORVON_IOINHI,IORVOFF_IOINORM)
Q
Not all the VistA code is this bad; some of it is worse.
That said, the DoD alternative (AHLTA) – and in fact the most well-known proprietary alternative – all share the same obscure programming language (MUMPS) and are almost certainly equally bad – but since their code is not public, it’s harder to critique them. In fact, I would expect them to be worse since they were not written with collaborative development in mind.
Say all the nice things about MUMPS you want: In the end, the choice of an ugly, archaic technology will decrease interest in any project by prospective contributors, thus decreasing the value of the collaborative model. This is Technical Debt we may not be able to repay.
Interestingly, Philip Newcomb, CEO of The Software Revolution Inc., has asserted that his company’s technology could convert VistA from MUMPS to J2EE in about a year for $10m. If true, this would be a bargain, IMHO. I’ve met with Phil twice in the past decade, and his claims are impressive, although I have no personal experience with the results of his company’s work. Other companies perform similar services – Hatha Systems for example claims to do automated analysis of MUMPS.
Tom Munnecke, one of the original architects of VistA has eloquently defended the MUMPS database design, pointing out that medical data is rarely suited to the structured, SQL-esque approach of relational databases. He makes some fabulous points, and I actually think that he’s right in that many of the decisions that were made for MUMPS and VistA were remarkably prescient. It’s taken the rest of the IT world three decades to rediscover document-structured non-relational databases, now known collectively as NoSQL. Now that there’s a huge-amount of energy and expertise focused on large-scale non-relational data stores, maybe we should consider how to use that talent and energy for EHR?
In short, I think the VA should keep doing the OSEHRA thing to consolidate and modernize VistA. There’s two threads to that: first, they need to consolidate into an enterprise version of VistA for the VA to bring together the forks (which they are doing), and second, they should refactor and modernize the architecture and the tooling (which they claim they are doing). I am suspicious that they aren’t being bold enough, but I don’t know.
I remain skeptical that VistA can survive in the long term as a vibrant, community-driven open source project, if it continues as it is. In order to make VistA a viable project, the current MUMPS-based database need to be replaced with a modern NoSQL datastore of some kind, and the hyper-abbreviated MUMPS code needs to be replaced with something readable and maintainable. A colleague of mine (David Wheeler) once pointed out that MUMPS code doesn’t have to be unreadable, but the coding practices of VistA do not lend themselves to readability. A bold step would be to re-write the thing in Java or some other modern language; a mimimalist step would just re-write it in MUMPS that doesn’t suck so much. In David’s words:
I think you should note another alternative as well: Keep MUMPS, but translate the current MUMPS into readable code.
Yes, you’d still be working in an uncommon language. But that transformation would be especially trivial to do (and trivial to automate), and the risks from auto-translation would be far lower because the underlying environment and assumptions would be unchanged.
It seems to me that the big problem here isn’t really MUMPS, it’s the way MUMPS has been used. Developers have used MUMPS’ “you can abbreviate anything” combined with “use bad names”, which perhaps made sense 30 years ago but is a bad idea today. But you do not *HAVE* to create ugly code in MUMPS.
Using the Wikipedia MUMPS article example, here’s some line noise:
hello() w "Hello, World!",! qBut here’s legal MUMPS – it’s the same code, but unencrypted:
hello() write "Hello, World!",! quitI have no idea if translating to another language would be a better trade-off than translating it to readable MUMPS. But it’d be easy to hire somebody to briefly investigate the options, pros, and cons, so that a reasonable decision (based on EVIDENCE) could be made. And I think, in fairness, that alternative should be considered.
I also have concerns about the governance model of OSEHRA, but, as this blog post is already too long, I’ll save that for another article.
As promised at the top: Jon Stewart on AHLTA vs VistA:
Alas, this video is no longer available on Comedy Central. It was from March 2013, and was hilarious but tragic. If you can find it, post a link in the comments.