Thursday, November 29, 2007

Cool Technology of the Week


Have you ever wanted to record MP3's of voice or a musical instrument? Since the iPod does not do this, the easy way to record sound is to buy a portable MP3 recorder such as the Edirol R-09, M-Audio MicroTrack or Zoom H2.

However, then you're stuck with yet another gadget that does only one thing. Why not use your existing laptop for this purpose?

I tried it and the sound quality was so bad, that I was ready to give up. My audiophile friends explained that typical microphones really need a pre-amp before plugging them into a laptop. However, I did not want to spend the money on a high quality microphone plus pre-amp.

I discovered an entirely different technology that worked perfectly - USB Microphones. These devices include a high quality analog microphone, a pre-amp and an analog to digital converter. They plug directly into a USB port and do not use the audio components of your computer at all! Here's an overview of the technology and the leading products

I purchased a Blue Snowball Microphone for $100 and was amazed by its quality. I used the Snowball with my Mac using Garage Band, my Ubuntu Linux laptop using Audacity and a Windows XP laptop using Sound Recorder. Here's a sample of my Shakuhachi Japanese Flute recorded with the Snowball.

The Blue Snowball USB Microphone - truly a Cool Technology

In my Cool Technology of the Week entry two weeks ago, I discussed the Orb. This week, our Orbs went live. Per this picture, we've placed an Orb on the CEO's desk and linked it to our Emergency Department waiting room volume metrics.

The Orb supports 35 different colors and glows Blue if no patients are waiting, Greens for 1 to 5, Yellows for 6 to 10, Reds for 11 to 20 and Flashing Red for over 20. We were able to create this fully automated "glance-able" interface in one day by simply repurposing existing Service Orientated Architecture (SOA) tools already deployed in our clinical systems. If it is successful in the CEOs office, we'll add additional Orbs and a menu of performance metrics to track.

Tuesday, November 27, 2007

The Cost of Information Technology

I'm often asked how about my budgets for hospital information systems. In the interest of transparency here it is, in narrative and spreadsheet form.

Typically, Academic Health Centers spend about 2.5% of their operating budget on Information Technology. Our budget is between 1.8-3.1% depending on what you include. Overall, we receive $37.1 million out of BIDMC's $1,195 milllion dollar budget (3.1%) but we provide services for many non-BIDMC entities which are charged back. We also provide Media Services/Telemedicine, Library Services, Telecommunications and Health Information management services (medical records). If you remove these items to create an apples to apples comparison with other academic health centers, the percentage drops to 1.8%. These percentages are mixed blessing. Although it's laudable we can do so much with under 2% of the operating budget, there are clearly opportunities to enhance operations via additional IT expenditures.

The narrative is a breakdown of direct and indirect IS costs. Important to note - it includes fringe benefits, leased space costs and utilities, and capital depreciation on IS related items. It also includes the cost of services provided to non-BIDMC sites who we charge for our service. The grand total is $59m; much higher than our net operating budget. To properly represent the cost of an application such as Peoplesoft or Email, you really need to include these expenses. At the bottom of the table, I show the revenue recovered from other entities and address some of the typical questions often asked about operating and capital comparisons.

The spreadsheet shows the IS expense for FY08 for each of our cost centers. This includes a breakdown of fringe, space/utilities, and depreciation. We depreciate everything on a 5 year basis for purposes of the allocation.

This year's $169.4m BIDMC capital includes $57.8m for our new Center for Life Sciences Research building and $23m for backfilling vacated space as researchers move into the new building. The $15.4m IS capital budget also includes $3.4m for the Center for Life Sciences build out of telecom and network services. If you remove these for both BIDMC and IS, we are 13.5% of the total. The remainder is $8.5m for infrastructure, $.5m for Peoplesoft, and $3m for Disaster Recovery.

The most difficult variable to express when comparing institutions is the quality of information systems. Quality for IS comes through reliability, fast transaction speeds, application breadth and depth, and customer satisfaction. At present, I believe that the capital and operating budgets I've provided here will result in more than 99.9% reliability, under 2 second transactions speeds in all applications, reasonable progress on our effort to be 85% electronic in all our care areas, and high customer satisfaction. As I often say, providing Information Systems is a continuous journey, so we will continue to seek additional budgets to support ever evolving user requirements.

Monday, November 26, 2007

Devices for Bedside Computing

We're currently evaluating several devices for mobile bedside computing including subnotebooks (Dell D420, Lenovo X61), tablets (Motion Computing LE1600), and small form factor devices (the Apple iPhone/iPod Touch, Emano-Tec MedTab ). I will write another blog entry on stationary bedside computing devices such as wall mounted thin client devices.

Our early results are that nothing on the market yet meets our ideal requirements:
8 hour battery life
Can be easily disinfected to prevent the transmission of disease
Can be dropped 5 feet without significant damage
Under 2 pounds

Here's a list of those machines which attempt to fill the niche for mobile bedside computers

DellD420
Battery Life - 3 hours
Ability to disinfect - poor
Drop resistance - fair
Size/Weight - 8.3 x 11.6 x 1/ 3.0 pounds

Lenovo X61
Battery Life - 6 hours
Ability to disinfect - poor
Drop resistance - fair
Size/Weight - 8.3 x 10.5 x 1.4 / 3.6 pounds

Motion Computing C5 (Also known as the Intel Mobile Clinical Assistant)
Battery Life - 3 hours
Ability to disinfect - excellent
Drop resistance - good
Size/Weight - 10.0" x 10.0" x .95” / 3.3 pounds

Motion Computing LS800
Battery Life - 3 hours
Ability to disinfect - good
Drop resistance - good
Size/Weight - 8.94” x 6.69” x 0.87”/ 2.2 pounds

Motion Computing LE1700
Battery Life - 3 hours
Ability to disinfect - good
Drop resistance - good
Size/Weight - 11.65" x 9.64" x 0.74” / 3.3 pounds

Apple iPhone/iPod Touch
Battery Life - 3 hours
Ability to disinfect - good
Drop resistance - fair
Size/Weight - 2.4 x 4.5 x .46/ 4.8 ounces

Emano-Tec MedTab(a startup which is just entering the market)
Battery Life - 24 hours
Ability to disinfect - good
Drop resistance - good
Size/Weight - 5.5" X 7.5" X 0.5"/12 ounces

I'm personally carrying around the Dell D420 and Lenovo X61 to assess their reliability and durability.

We're piloting 5 LE1700's in our Emergency Department. The LE1700 has a 12.1" screen size compared to a 10.4" in the C5 and 8.4" i the LS800. We're testing it with a medication reconciliation application and will soon know more about its support of clinician workflow. In general, all the Motion Computing devices seem rugged, relatively easy to disinfect, and well engineered. However, battery life is limited to 3 hours, so we'll need to keep charging stations handy throughout the department.

The Emano-tec MedTab form factor and battery life are ideal. It uses an eInk display just like the recently announced Amazon Kindle. The lack of color may be a major limitation for some applications. At present, the device is not available in large quantities, so we're testing a prototype, also in our Emergency Department.

The iPod Touch is promising. A slightly larger form factor and better power management (longer life and/or hot swappable batteries) could make this an ideal medical device. Apple's attention to human interface features really shows here, but it's clearly intended to be a device for personal, not business use. We're testing it with our Emergency Department dashboard application.

Size and battery life for laptops is not as relevant if used as a Computer on Wheels on carts. We use two kinds of carts - The unpowered Ergoton StyleView and the powered Infologix SL Ultra Cart

The hardware and software development efforts of the past few years are getting us closer to the ideal mobile clinical device. At BIDMC one challenge is the difficulty involved in secure authentication. We use secure passwords (requiring capital letters, numbers, symbols, etc) which are difficult to rapidly enter via a tablet. Solutions include biometrics (we are testing Omnipass) and novel login mechanisms such as graphical authentication.

More to come as we complete our evaluation by the end of the year. I welcome any comments on other's experiences with mobile bedside computing devices.

Are Regional Health Information Organizations (RHIOs) Sustainable?

As CEO of the Massachusetts Regional Health Information Organization (RHIO) called MA-SHARE, I have spent 4 years working with stakeholders in Massachusetts to create a sustainable business model for health information exchange.

MA-SHARE’s first effort in 2004 was the MedsInfo project, a state-wide medication history exchange pilot based on payer claims data. We learned a great deal about privacy, workflow, data expectations, and health information exchange operations. The project was terminated after the pilot because participants were not ready to fund the true cost of ongoing operations given the lack of integration of the data into clinician workflow and the inherent incompleteness of the data (only 66% of patients had medication data in our regional payer databases as of 2004). As of 2007, the two largest national e-Prescribing exchanges, RxHub and SureScripts, have much more complete networks and we've integrated the former MedsInfo functionality into our e-Prescribing utility, described below.

In 2005 and 2006, working with the Markle Foundation and the Office of the National Coordinator, we developed a Nationwide Health Information Network implementation pilot based on a state-wide master patient index called the Record Locator Service. The pilot demonstrated the value of the emerging clinical data exchange architecture to support provider-to-provider data exchange, personal health records, and biosurveillance. The architecture worked well, but the project was terminated after the pilot because participants were not ready to fund the true cost of ongoing operations required to maintain the Record Locator Service.

In 2006 and 2007, we implemented a state-wide e-Prescribing gateway. We've transmitted over 100,000 electronic prescription transactions through our exchange and we are live with formulary enforcement, eligibility checking, dispensed medication history including drug/drug interaction checking and routing to retail/mail order pharmacies. The stakeholders have found value in paying for the cost of ongoing operations of this infrastructure since it reduces costs to the payers by enhancing the use of generics/formulary medications, it reduces costs to pharmacies by eliminating paper workflows and it improves workflow for providers by streamlining renewal workflow. We've implemented our e-Prescribing gateway at CareGroup, Partners and soon Children's Hospital. We will work in 2008 to expand the use of the gateway to connect to vendor systems such as Cerner and Meditech, as well as to encourage its use in more institutions.

In 2007, we implemented our "push pilot" using national standards to share discharge summaries and emergency department summaries among caregivers. We use the same software application that routes prescriptions between providers and pharmacies to securely route documents provider to provider. This clinical data exchange approach is truly low cost and simple. All that is required is a sender which can summarize tabular and narrative data in the format specified by HITSP and an organization which can receive this data via direct integration into an electronic health record, secure email or fax. Cerner, MEDITECH, eClinicalWorks and GE Centricity are among the EMR vendors supporting the design and implementation of this project. We are optimistic that the value to the stakeholders of exchanging clinical summaries will be sustainable based on cost avoidance. By eliminating the expense of chart copying, mailing, and paper-based record storage, hospitals seem willing to fund health information exchange of summaries out of projected cost savings. It's also a great political win for the hospital, since pushing clinical summaries keeps the primary caregivers and referring physicians well informed, enhancing their satisfaction. It provides care continuity by ensuring all caregivers (inpatient, outpatient, Emergency Department, rehabilitation, and long term care facilities) are given a consistent medication list, problem list, laboratory summary, and discharge narrative. As personal health record services such as Microsoft HealthVault, Google's Health efforts and Dossia through Indivo Health are more widely deployed, we may also push data directly into personal health repositories at patient request.

MA-SHARE’s budget in 2008 is approaching the same kind of sustainablity we've achieved with our financial data exchange, NEHEN. All 'lights on' operations are funded by the stakeholders plus $250,000 is available each year for new projects and enhancements. No grant funding or soft money source will be used in 2008. Our hope is that more stakeholders will sign up to participate in MA-SHARE over time, further funding research and development of high value health information exchange products for our community. The big lesson learned in our statewide initiatives, MA-Share and NEHEN, is that grant funding and large stakeholder (academic medical centers/payers) contributions precede sustainability. To achieve sustainability, the initial efforts must be expanded to meet the needs of the common marketplace. We believe our push model addresses this issue.

Health Information Exchanges in the US are in tenuous financial shape. We've been exploring sustainable business models in Massachusetts for 4 years. Many RHIOs still depend on grants, which eventually end and thus are not a good business model. I believe that Health Information Exchanges will evolve to meet the local business needs of many communities but that a nationwide health information network linking together these local exchanges will not be widely deployed until more consistent funding is available. In many ways, data exchange is a public good, which is hard to support entirely from local stakeholders. Additional funding from federal and state sources would help. The level of investment in healthcare information exchange in Canada and the UK far exceeds that in the US. I hope that Bush's 2004 commitment to have every clinician in the country wired by 2014 will be met with increases in funding to support it.

Sunday, November 25, 2007

Quality Metrics for the Country

I recently had the privilege of participating in the Health Information Technology Expert Panel (HITEP) of the National Quality Forum (NQF), part of a joint effort of many stakeholders to rethink the way quality is measured from data in Electronic Health Records and Hospital Information Systems.

This week, the draft report will describe an analysis of data types commonly used in quality measures, as part of the quality metric "harmonization" process to be implemented by HITSP, NQF, AMA/NCQA and the Quality Metric authoring organizations (Joint Commission, AQA, HQA etc.)

The idea is simple. In the past, quality metric authors have produced carefully specified, evidence-based quality measures. The challenge is that these carefully designed indicators are not computable from existing hospital information systems and ambulatory care records. There are so many exclusions in the measurements that manual chart abstraction, an expensive and time consuming approach, is the only way to collect data for these measurements.

For example, excluding all patients on “comfort measures only” from quality measures creates a very accurate denominator, but no hospital information system in production today uses the SNOMED vocabulary term for “comfort measures only” in the electronic patient record. This begs the question - if this exclusion were eliminated would it really matter? If we assume that “comfort measures only” is a reasonably evenly distributed event at all hospitals in the country, then the metrics will be very slightly off for every hospital. Similarly, exclusing women with a history of polycystic ovarian syndrome from diabetic measures can be challenging. Assuming polycystic disease is evenly distributed in the country, why use this exclusionary criteria?

With the idea that we should create the best quality measures possible given the data we can gather electronically, here's the new process. As part of the NQF-endorsement process, Quality measure developers will submit their proposed measures to the NQF. The NQF will analyze the data types needed to complete the measures (i.e. labs, medications, problems, allergies, demographics) and forward a request for standards harmonization to HITSP. HITSP will recommend the standards for each of these data types and will report on gaps. Further, HITSP and NQF will work together to report on data quality/adoption - even with good standards, how much of the data is available today with reasonably good data integrity. Based on the analysis of data types, standards readiness, and data avalability analysis, NQF will offer feedback to the quality measure developers to refine the measures to ensure they are computable.

With easily calculated quality metrics, hospitals and ambulatory care facilities can deploy real time dashboards and decision support to offer clinicians the just-in-time information they need to improve quality. Compare that to manual chart abstraction which takes months and $20/chart in labor.

As a first step, NQF and HITSP have worked through 84 NQF-endorsed AQA/HQA measures that exist for the IOM Priority Areas listed below:

1. Asthma
2. Cancer screening
3. Care coordination
4. Diabetes
5. End-of-life with advanced organ system failure
6. Frailty associated with old age
7. Immunization
8. Ischemic heart disease
9. Major depression
10. Medication management
11. Pregnancy and childbirth
12. Stroke
13. Tobacco dependence treatment in adults

and derived 35 data types requiring standards. The HITSP Population Health Technical Committee will deliver the harmonized standards for these data types on December 13, 2007 at the HITSP Panel meeting in Washington.

Working together, NQF/HITEP, HITSP, AMA/NCQA, and the quality metric authoring organizations are well along the way to producing computable quality measures for the country. As these are implemented in hospitals and ambulatory care centers in 2008-2009, the burden of data collection will be reduced and the amount of actionable knowledge (see my previous blog entry here) about the care we deliver will markedly increase.

Wednesday, November 21, 2007

Cool technology of the week

Last week I started a Cool Technology post which I intend to do every Friday.

As part of my flexible work arrangement research, I've been testing video conferencing technologies. The coolest technology is Cisco's Telepresence infrastructure. John Chambers demonstrates it here

I've tested H323 software and hardware, Video IM chat, and iChat via Jabber. The big question raised by all of these technologies is best described in this email sent to me by Paul Gray, Professor Emeritus of Information Science, Claremont Graduate University:

"Being retired, I receive my copies of Computerworld in batches from my office. Hence I only now read your September 15 article on flexible schedules. I was pleased to see that you found the need for initial meetings important in your thinking. I thought you would like to know that this concept is not a new idea. When we first proposed telecommuting (Telecommuting-Transportation Tradeoffs: Options for Tomorrow, Wiley 1975) we quoted results that we found in the literature on the dispersal of government workers out of central London and central Stockholm in the 1960s. The dispersal was the result of, for example in London, of the concentration of office jobs that wound up depopulating the hinterlands of young people. Everybody complained that they could not be moved out because they needed continual face to face contact with people in other agencies. Studies were done that found that once there is an initial meeting, which coupled a human face and body language with voice and correspondence, people were able to work in dispersed mode with no loss of effectiveness. However, they did need periodic (typically 6 month) refreshing of the initial contact so that the ties would be maintained"

It may turn out that audio via phone, combined with desktop sharing/whiteboards/IM will be good enough without video. My full report will be complete in January.

A Vegan Thanksgiving



People have asked me what a vegan eats for Thanksgiving. Today's menu includes

Salad: Warm heirloom beet, carrot and edamame salad - a simple mixture of freshly boiled vegetables placed on a bed of baby greens and drizzled in balsamic vinegar

Protein: Tofurky - a tofu and grain-based roast available from Turtle Island Foods . I do not typically eat meat substitutes since I enjoy the inherent food qualities of tofu, tempeh and seitan, but a Tofurky is great for family holiday entertaining.

Sides:
Harvest vegetable medley -brussell sprouts, butternut squash, cauliflower, fingerling potatoes, leeks, carrots, parsnips, garlic, bell pepper and fresh herbs roasted at 450F

Red lentils - simmered in vegan low-sodium bullion and flavored with cumin

Fresh roasted chestnuts

Mashed yukon gold potatoes (no butter or cream added, just a bit of soy milk)

Wines: Louis Roederer non-vintage Champagne, Chassagne Montrachet Maltroye 2001

Dessert: Vegan cranberry apple pie, Gyokuro Asahi green tea

I really look forward to those fresh Tofurky sandwiches after Thanksgiving!

Although the focus of this blog is IT, here's a brief personal word about my own vegan experience. In the 1990's I had a body mass index of 30 (that's obese) . I lived the all stress diet of a CIO: Starbucks grande lattes, supersized fast food, and business meals at the local steakhouse. I kept a bottle of Tums and Motrin on my desk. In 2000, I visited my primary care physician for the first time in a decade and discovered that my HDL cholesterol was low, LDL cholesterol was high, and blood pressure was borderline hypertensive (140/90).

I was offered the possibility of Lipitor, Ace Inhibitors, Beta Blockers, Nexium etc. Instead I changed my lifestyle. I first eliminated caffeine. Other than two weeks of headaches, sweats, irritability, tremulousness, insomnia, and fatigue, it was no problem. Without caffeine, I no longer craved the afternoon Krispy Kreme and all gastric reflux symptoms disappeared. My emotions and blood glucose became stable 24 hours a day. I no longer craved morning coffee and slept naturally instead of waking to caffeine withdrawl.

I then stopped eating fast food, fried foods, and beef. From there, I ate only chicken and fish. Then, I eliminated all meats and fish. As my 14 year old daughter would say, "I no longer ate anything with a mother". From there, I eliminated all animal products - no dairy and no eggs. I ate tofu, fresh vegetables, rice, and legumes, losing 65 pounds (before and after). That was 5 years ago and the drop in weight has enabled me to become more active. Today, I do not miss animal products. However, when I travel (especially in Eastern Europe) being vegan is challenging, so I may have vegetarian meals that include eggs or dairy. The end result of my years as a vegan is the data from my most recent primary care visit:

October 2007
Body Mass Index 21
Cholesterol (mg/dL) 130
Triglyceride (mg/dL) 80
HDL (mg/dL) 47
LDL (mg/dL) 60

BP Systolic (mm Hg) 116
BP Diastolic (mm Hg) 72
Pulse rate (/min) 53

No Tums, no Motrin, no medications of any kind. I have extended my vegan lifestyle to eliminate animal products from my wardrobe, including all leather. My current clothing is entirely rayon/polyester with a bit of cotton, and shoes made from polyester microfiber. I feel I've done my part to reduce my carbon footprint, minimize my healthcare costs, and maximize my longevity. I do not attempt to inflict any philosophy on my friends and colleagues, but I can recommend vegetarianism/veganism. It worked for me!

Tuesday, November 20, 2007

What will keep me up at night in 2008

Every year, I have new infrastructure and application challenges. In 2002, I had an outage that required a major focus on replacing the entire data network. In 2007, a number of safety/quality related projects including medication reconciliation and automated chemotherapy ordering were my focus. What about 2008?

1. Stark safe harbors now enable hospitals to fund 85% of the implementation costs of electronic health records for non-owned physicians. I will be implementing a large hosting facility offering web-based electronic health records for 300 private physicians in New England.

2. Storage is increasingly an utility. This means that heat, power, light, networking and terabytes need to be provisioned on demand. Achieving a balance of highly reliable storage, archival storage, and backup at low cost will be a major body of work over the next year. This means I'll have to figure out the right combination of continuous data protection, hierarchical storage management, data de-duplication, virtual/actual tape libraries, and information lifecycle management. Of course we'll need to implement this new infrastructure in a "green" manner that keeps the entire power consumption of our data center under 220kw, our 2008 energy goal.

3. e-Prescribing means much more than just prescription routing. It means eligibility checking, formulary enforcement, community medication history sharing and decision support. We will complete the rollout of all of these features to all of our clinicians this year. We'll also have other medication safety initiatives are part of our pay for performance contracts including electronic medication administration records and protocol driven inpatient chemotherapy systems.

4. Data sharing for clinical care among a community of caregivers poses significant privacy policy and technology challenges. We are going live with clinical summary sharing using the Continuity of Care Document among the providers of BIDMC, Lahey, Children's and Northeast Health Systems in 2008.

5. Security is a journey that will require enhanced virus/malware protection, web content filtering, host-based intrusion protection, and intelligent audit trail reviews. Substantial staff resources will be required to safeguard patient confidentiality.

6. RFID and Bar coding will increasingly be used to identify patients, staff, medications and assets. Workflow will be driven by the proximity of patients, doctors, and supplies. Deploying the right technology for the right purpose will require several pilots.

7. Providing decision support to every level of the organization will require additional tools and staff. Quality improvement, outcomes measurement, pay for performance goals, and clinical research necessitate more analysts, data marts, and self service applications to supply information on a need to know basis.

8. Compliance requirements for new revenue cycle workflows including enhanced electronic data interchange for claims, national provider identifier support, and evolving coding methodologies will require substantial improvements to existing systems.

9. Internal and external websites need to be enhanced to support self-service publishing models, collaboration and new media. This means new content management systems, enhanced wikis/blogs/forums/whiteboards, and search engines.

10. Disaster recovery needs to be built into the design of every application. Recovery time must be on the order of hours and the recovery point objective is 100% data integrity. The only way to achieve this level of reliability is to have entirely redundant data centers.

For a list of the 200 different projects that will enable us to meet these goals, you'll find my 2008 BIDMC IS Operating plan here and my 2008 Harvard Medical School IS Operating plan here.

Monday, November 19, 2007

How to say "No"

I was recently asked to give a lecture about how I say "no" to new project requests. Of course I have governance committees which help prioritize all IT projects based on

Return on Investment
Quality/Compliance
Impact factor - number of doctors, nurses, staff and patients who will benefit
Alignment with the strategic needs of the business

Beyond my governance processes, which I will describe further in another post, my top 10 list of how to say "no" is more about people than prioritization.

10. Select your change (and what not to change)
I've learned that my hospital organization (BIDMC) does not readily accept off the shelf enterprise application software. In the past decade, we've stopped a major clinical and a major revenue cycle project because of the limited customization possibilities with vendor supplied software. To this day, our self-built customized enterprise applications keep customers happy at low cost. Of course I still buy many departmental systems (lab, critical care, anesthesia, labor and deliver monitoring, PACS, cardiology) but will no longer try to replace our enterprise clinical applications with vendor products. This is an automatic "no" that customers understand.

9. Identify those who will lose and take them to lunch
On a given day, 10% of the organization is not completely satisfied with the triage decisions made by my governance committees. In a world of limited supply and infinite demand, the organization needs to say "no" to many projects. I find that bad news does not travel well via email, hence personal contact is needed to explain many prioritization decisions. I try to make personal contact with those whose projects are not funded/prioritized. Whenever possible, instead of "never", I say "not now" to lower priority projects.

8. Acknowledge the loss
Many people will accept change if the process is transparent, they are involved in the decision, and their losses are acknowledged. Telling folks that you understand the impact of negative decisions and expressing a willingness to work together in the future goes a long way.

7. Over Communicate
Rumors are often worse than the truth. Every Friday I send out a broadcast email to the entire organization explaining issues, good news, bad news, and future plans.

6. Be Honest and Consistent
I work hard to tell all stakeholders the same message. If everyone hears the good and bad news consistently, the credibility of IT is enhanced.

5. Consensus is not essential
A vote of 500 to 1 is not a tie. If governance works objectively, even politically powerful stakeholders cannot veto prioritization decisions which are in the best interest of the organization.

4. Embrace conflict
Sometimes the right decisions are the hard or politically challenging ones. By expecting conflict every day, the CIO can make decisions more dispassionately. My training as an emergency physician prepared me to approach every situation with balanced emotions. Eliminating caffeine 5 years ago helped too.

3. Focus on your detractors
Sometimes organizations can be 1000 points of veto. By focusing on those who oppose projects instead of those which support them, I can use my time most effectively. I'd rather meet with my friends, but my day is optimized when I spend the day with my detractors. Sometimes detractors become friends, but at least all detractors understand the rationale for "no" decisions.

2. The last two minutes of the meeting are the most important
It's very common for politically challenging meetings to end with differing opinions as to what was discussed. Using the last two minutes of the meeting to review all the decisions made and next steps, then memorializing that conversation in written minutes, enhances the communication of "no"

1. You cannot please everyone
I accept that the good of the many outweighs the needs of the few, even if I have to be the "no" guy.

Friday, November 16, 2007

Our Secure email strategy

Here's the third in my series on providing secure, spam free, virus free email to 25,000 users.

Regular email that travels over the internet is completely insecure. It's about the same as sending a postcard, since any server administrator or network expert could intercept and read mail as it goes from sender to receiver.

For healthcare email, especially messages containing protected health information (PHI), secure email between organizations is a best practice consistent with the spirit of HIPAA. Over the past 5 years, many organizations in the Massachusetts healthcare community have implemented secure email for all traffic between organizations.

We started the process the in 2002 by working with leading vendors of messaging systems to harmonize the standards used to send email. We used the Internet Engineering Task Force's (IETF) Request for Comment (RFC) process to complete the specifications for S/MIME Gateways. Since that time, another approach called Open Pretty Good Privacy (OpenPGP) has also become popular in the messaging market. A comparison of these two standards is here

Today, Beth Israel Deaconess Medical Center, Children's Hospital, Tufts Health Plan, Harvard Pilgrim Health Plan, and the Division of Medical Assistance (Medicaid) are using gateways that support S/MIME and OpenPGP. Here's how it works. Someone at BIDMC sends an email from a web browser or their email program and it passes into our Microsoft Exchange email server. Between Exchange and the internet, we've inserted the Tumbleweed Secure Messenger. It has a list of all our business partners with secure email systems. If an email is sent to one of these partners, the email is encrypted and sent to the receipient's secure email gateway. Other emails are sent insecurely via the regular internet methods. We are also piloting content filtering systems that identify insecure emails containing credit card data or patient identifiers so we can quarantine those before they are sent over the public internet.

The best feature about this secure email approach is that users do not even know we have it in place. Security is organization to organization, not person to person, so no special email clients or digital certificates are needed. It's seamless, effective and low cost.

As you can tell from my last three posts, supporting spam-free, virus free email is a major undertaking.

Thursday, November 15, 2007

Supporting electronic health records for non-owned doctors

Implementing electronic health records requires transformation of a medical practice. It's more about workflow and change management than technology. In Massachusetts, competitive pressures, pay for performance contracts, and increasing demands from specialists to be connected to primary care givers are motivating clinicians to install electronic health records. The challenge is how to pay for them and how to provide the services necessary to ensure successful implementation.

Recently, Stark regulations have been changed to enable hospitals to fund up to 85% of the implementation costs of electronic health records for non-owned clinicians. This means that hundreds of clinicians in the community are now looking to BIDMC as an information technology provider. Although we oversee 200 trillion bytes of data, 16000 network connections, 8000 personal computers, and 150 applications, it's challenging to scale our team to support hundreds of new physician practices in the community.

How will we do it?

We've partnered with a dream team of collaborators to build a robust, cost effective, web-accessible electronic health record hosting facility that goes live in early 2008. Our partners include

eClinicalWorks - a leading provider of practice management and CCHIT certified electronic health records, accessible over the inernet using a smart web client, from anywhere in the world

Concordant - a leading provider of desktop, network, and server hosting services for clinician offices throughout our region.

MassPro - a regional peer review organization which provides best practice templates for physicians implementing electronic health records and which created DOQ-IT University, a training program for professionals implementing physician automation. It's also a leader in quality measurement

Massachuetts eHealth Collaborative - our regional implementer of electronic health records with expertise in practice transformation.

Third Brigade - an expert internet security firm, which will audit all our systems to ensure we are protecting patient confidentiality.

Working together with these collaborators, BIDMC will offer a suite of services that enable us to expand the scale of our current IS operations into the community. My team and I will continue to provide the medical informatics expertise, decision support knowledge, and interoperability standards, but our team of partners will help us execute the vision in a way that enables us to balance time, scope and resources.

Wednesday, November 14, 2007

Cool technology of the week

I'd like to start a new feature of my blog, which will appear every Friday. It's called "the cool technology of the week". Every week I meet with vendors, pilot new devices, and hear from my staff about the latest information technology products. When I find something that has potential, I'll describe it here.

My first cool technology of the week, is the Orb, a "glance-able" user interface available here. The concept is simple. The Orb is a handblown etched glass sphere containing LEDs for every color of the rainbow plus a text pager interface with an XML parser. Just plug the Orb in any geography in the US that's covered by a paging service. Now the creative part begins. Would your senior executives like to be informed about quality metrics, financial metrics, or workflow metrics? Just pick a color, write a simple interface and you're done. For example, when Emergency Department waiting times are under an hour, he Orb could glow green. When times are between 1-2 hours, the Orb glows yellow. Between 2-3 hours, the Orb glows red. Over 3 hours, flashing red. Place orbs at the nursing stations, in waiting rooms, or on the CEO's desk etc. Metrics are turned into a web service call that results in a page to the Orb every 5 minutes, updating the color.

I've ordered one and I'll be placing all my IT uptime metrics on an Orb feed. At the front desk of Beth Israel Deaconess, you'll soon see a "glance-able" dashboard of the current state of all our applications and infrastructure.

Tuesday, November 13, 2007

My top 10 rules for Email Triage

This is the second in my series about email.

I receive over 600 email messages each day (with virtually no Spam, so they are all legitimate) and respond to most via Blackberry. How do I triage 600 messages? I use these 10 rules to mentally score each email:

1. E-mail marked with a “high importance” exclamation point must pass the “cry wolf” test. Is the sender a habitual “high importance” e-mailer? Are these e-mails actually important? If not, the sender's emails lose points.

2. I give points to high-priority people: my senior management, my direct reports, my family members and my key customers.

3. I do the same for high-priority subjects: critical staff issues, health issues and major financial issues.

4. I rate email based on the contents of the “To,” “cc” and “bcc” fields. If I am the only person in the To field, the e-mail gets points. If I am in the To field with a dozen other people, it’s neutral. If I'm only cc’d, it loses points. A bcc loses a lot of points, since I believe email should always be transparent. E-mail should not be used as a weapon.

5. I penalize email with emotional words, capital letters or anything less than civil language.

6. I downgrade email messages longer than five BlackBerry screens. Issues that complex require a phone call.

7. Email responses that say only “Thanks,” “OK” or “Have a nice day” are social pleasantries that I appreciate, but move to the bottom of my queue.

8. Email with colorful backgrounds, embedded graphics or mixed font sizes lose points.

9. I separate email into three categories - that which is just informational (an FYI), that which requires a short response and that which requires a lengthy, thoughtful response. I leave the lengthy responses to the end of the day.

10. More than 3 emails about a topic requires a phone call or meeting. Trying to resolve complex issues via and endless ping pong of emails is inappropriate.

These 10 rules really help me navigate my 600 emails each day.

If we actually automated the rules above and senders realized that their e-mail had to be truly relevant to get read, folks might think twice before pressing Send. The less important matters can wait until the next staff meeting. With some enforced discipline, we may be able to learn how to better communicate with one another more effectively and get back to our creative work.

One more truly controversial idea - Companies that send bulk e-mail should be forced to pay before an e-mail gateway delivers their mail. How many newsletters have you really “opted in” for? A micropayment fee system will keep companies honest about their opt-in and unsubscribe policies by aligning financial incentives.

The War Against Spam

In my earlier post about IT security, I described the Cold War between hackers/crackers/spammers and IT departments. Spam control is one of my most challenging battlefields.

Whenever I speak about security, I describe it as a Cold War between hackers/crackers/spammers and Information Technology departments. Spam control is one of our most challenging battlefields.

At BIDMC, we receive an average of 886,674 emails every day from the internet. We deliver 57,103 of these, meaning that 829,751 of these are Spam. This translates into 302,859,115 Spam per year or over a third of a BILLION Spam.

There are many commercial products on the market that can help with this problem. At BIDMC and Harvard Medical School we use Symantec Brightmail Anti-Spam Version 6.0. Here's the challenge - it's not easy to distinguish legitimate clinical email from advertising. In a medical environment our clinicians describe anatomy, medications, and diagnoses that might be the same key words used in emails which advertise herbals to enlarge your body parts. Suppose that our filters are tuned so tightly that all Spam is eliminated but also 1% of legitimate email is also blocked. The cost of this solution would be that 208,425 legitimate emails per year would be undelivered. Conversely, suppose our Spam filters are relaxed so that no legitimate email is blocked but also 1% of Spam gets through. The cost of this solution is that 3 million Spam make it to our inboxes every year.

The balance between false positives (blocking legitimate email) and false negatives (letting Spam through) is quite challenging and requires continuous updating of our Spam filtering techniques. We blacklist known spamming sites. We whitelist sites which send emails about anatomical parts, but are known clinical partners. We have a Spam Feedback mailbox which provides continuous feedback to Brightmail. We use Exchange and Outlook rules to automatically move Spam into folders. We block all ZIP files from the internet but notify recipients that an email containing a ZIP was received and blocked.

Two types of Spam still get through

1. Spammers embed graphics of advertisements instead of text. Since computers cannot read graphics, we cannot filter them

2. Spammers use words that are not unique i.e. "enhance your being a male" that cannot be filtered without removing legitimate email

At present, using Brightmail and the other techniques described above, we block 99% of all Spam (one third of a BILLION) and deliver nearly 100% of legitimate email, allowing 3 million Spam per year to land in our mailboxes but ensuring our doctors and staff get the mission critical email they need to deliver good care. We'll continue to enhance our Spam filtering systems, but you can still expect some Spam to get through. As fast as we innovate, spammers innovate, creating a continuous battle against Spam.

The ultimate answer may be that the internet email infrastructure itself needs to be revised to deny all email traffic except that which is specifically whitelisted by email servers and users. Earthlink and other ISPs have used this approach. It's a bit irritating for the sender who is told that email will not be received until the recipient approves the sender. It's a hassle for the recipient who has to approve every incoming email sender. The result however is that offending senders are blocked forever and no spam passes through the human medicated approval process.

Other alternatives are to charge bulk email senders postage for sending their contents over the internet, but that's tomorrow's blog entry!

Monday, November 12, 2007

Data, Information, Knowledge, and Wisdom


One of the challenges of being a doctor in the 21st century is information overload. More medical literature is published every year than a doctor can read in a lifetime. As electronic health records become more common, doctors can be overwhelmed with data gathered about each patient. Doctors do not want to review hundreds of normal findings, they want to know what is actionable for each of their patients to keep them healthy

Healthcare CIOs should implement applications which filter data so that it becomes information, transform information into knowledge, and ulimately provide clinicians with wisdom based on that knowledge at the exact time they need it.

Here's an example. Suppose a patient's blood pressure is 100/50. That's data. Suppose that patient has a ten year history of blood pressures of 150/100. That's information. Suppose that the patient has a known history of coronary artery disease and is now experiencing chest pain. The sudden drop in blood pressure could indicate a serious myocardial infarction in progress. That's knowledge. It's time to give the patient an aspirin, oxygen, and nitrates immediately. That's wisdom.

Recently, I asked my primary care physician to export my entire history from his electronic medical record system. Although I'm a completely healthy person, the result was a 77 page PDF. The document contains a mix of administrative and clinical data, numeric observations and unstructured text. It would take a physician about an hour to navigate all this data.

How can we turn this data into information? Over the past few years, my clinical information systems team, led by Dr. Larry Markson, has built "event driven medicine" into our applications. Events such as changes in medications, patient visits for diagnostic testing, lab results, or newly discovered allergic reactions generate data which can be transformed into actionable wisdom. Here are three examples:

When a doctor writes for a medication at BIDMC, a query is sent to our regional data exchange determining the patient's insurance coverage for pharmaceuticals. Based on the answer, we access the appropriate payer-specific formulary so that all medications are preferentially chosen to minimize cost and maximize effectiveness for each patient. Every prescribed medication is checked against the entire history of the patient's active medications from pharmacy and payer databases throughout the country. Safety issues, guidelines and best practices are displayed to the clinician, ensuring quality care. When the correct, safe medication in the right dose is selected it is instantly routed to the pharmacy of the patient's choice, going from the doctor's brain to the patient's vein without any handwriting or human interpretation. All of this happens in real time based on the data found in electronic health records, information about trends in body functions, knowledge from decision support databases, and wisdom from the orchestration of all these moving parts behind the scenes via interoperable web services, ultimately providing the best choice for each medication written. This week, we just completed our 100,000th medication processed this way.

When a doctor orders a radiology test a BIDMC, a query is sent to a decision support engine which we co-developed with Safe-Med. Over 1000 best practice rules from the American College of Radiology and the world's radiology literature are examined, along with patient medications, laboratories, allergies and demographics, to select the most appropriate radiology test based on evidence. Radiology exams are scored from 5 stars to 1 star (the image shown above), balancing efficacy, risk and cost. If a clinician orders one of these tests, a pre-authorization is sent to the payer in real time and the test is automatically approved. All of this happens in a few seconds, using patient data plus the knowledge from the literature to yield a wise choice for radiology diagnostic testing. 100% of high cost radiology tests are processed this way.

When a doctor identifies a chronic diease condition at BIDMC, a decision support "screening sheet" is created to track all the events in a patient's care. Diabetic tracking includes lipids, glucose, eye exams, foot exams, Hemoglobin A1-C, immunizations, and weight. Whenever an event occurs - such as a lab result or appointment, the screening sheet is updated and decision support rules recommend the best practices for diabetic care, filtering all this data into a concrete set of recommendations such as "patient is past due for an eye exam" or "patient should receive pneumovax this season". Clinicians do not need to focus on the raw data, instead they can review suggestions in real time to optimize the care of the patient. This year we achieved all our pay for performance goals using this approach.

Like many other projects, the pursuit of event driven medicine is a journey. Over the next few years, we'll continue our efforts to ensure that clinicians are given the real time wisdom they need to deliver safe, cost effective and appropriate care.

My Gadgets

I'm often asked what technology I own and why. Here's the rundown:

My personal laptop is a Dell D420 1.06 GHz Core Solo with 1 Gig of RAM running Ubuntu Gutsy Gibbon, Open Office 2.3, Firefox 2.0.0.8 and Evolution email 2.12 connected to Exchange 2003. I typically replace my laptop every two years because my 400,000 miles of travel each year are brutal on keyboards and screens. My criteria for laptops is simple: 2.5 pounds or less, 5 hour battery life, 12" screen so that I can use it in a coach airline seat, and no built in DVD/CD to reduce weight. The Dell D420 and the Lenovo X61 are the only machines that seem to currently meet these requirements. As I've said many times, I will migrate to an Apple subnotebook as soon as one that meets the specifications above is produced.

My home desktop is an iMac 20" 2.4 Ghz Core 2 Duo with 2 Gigs of RAM running Leopard, Open Office 2.3, Firefox 2.0.0.9, and Photoshop CS3. I replace my home desktop every 3-4 years to ensure hardware compatibility with new operating system and application releases. My criteria for a home computer includes the ability to manage personal media including video editing, digital photography, and self recorded mp3 (I play the Japanese flute), plus support for web browsing. The 20" screen is the perfect balance between readability and desk real estate.

My mobile device is a Blackberry 8707G worldphone with an AT&T Sim Card. The selection process for my mobile device was particularly challenging, since I wanted one device that could work on every square inch of the planet. My quest is described in my blog entry here

I drive a 2005 Prius with built in bluetooth capability. As I touch the car, it senses a bluetooth fob on my body and unlocks the door. I push the start button and begin driving. My Blackberry instantly bonds with the Prius, so that my phone is controlled by the car as a drive.

I own a iPod Nano loaded with a few dozen of my own Shakuhachi recordings

My printers all all HP - at the office, I use an HP LaserJet 1200 and at home I use an HP K550 network printer

I do not carry a pager, a bluetooth headset, or a personal GPS, although I do own a Garmin Etrex for Geocaching

I have an implanted RFID as described in my earlier blog post here

By limiting my technology to a Blackberry 8707G and a Dell D420, I limit the weight I carry on the road as well the risk of device theft or damage. I'm a strong believer in redundancy and I do carry a 1 Gig USB drive in my wallet containing all my recent hard drive contents, just in case!

Sunday, November 11, 2007

The Tyranny of the Urgent

In my recent blog entry "It's not a job, it's a lifestyle", I mentioned that I reserve 50% of my schedule for the work of each day and 50% for more longitudinal work. Explaining this balance justifies another blog entry.

Providing cutting edge clinical applications to a hospital is journey. It requires daily efforts to refine workflow and encourage adoption but it also requires a multiyear plan to ensure future needs will be met. Three years ago, it was clear that e-Prescribing would be very important. However in 2004, faxing to retail pharmacies was all we had. Today, with the Surescripts connection to retail pharmacies and the RxHub connection to mail order pharmacies, we have fully electronic transmission for 90% of our prescriptions. The only way we were able to support all aspects of e-Prescribing including formulary enforcement, eligibility checking, community drug history with drug/drug interactions, and prescription routing was to focus a portion of each day on our long term goals without being derailed by each day's distractions.

I call the distractions of each day, "the tyranny of the urgent". Everyone believes that long term goals like medication safety are essential, but each day some stakeholder has a new, mission critical project that is expected to trump existing priorities. Of course there are legitimate urgent projects that must be done for quality, safety, compliance or return on investment, but if we allowed every project proposed each day to defer our multi-year plan, we'd never accomplish our long term goals.

Thus, I spend 50% of my day on email, phone calls and the tyranny of the urgent, but reserve 50% of each day for those projects which will create revolutionary change three years from now. To do so requires two kinds of plans.

My Operating Plan outlines the goals of each year - ensuring that each business owner's strategic priorities are met on a yearly basis. To ensure stakeholders understand the major themes of our yearly operating plan, I produce a thematic summary. The FY08 summary can be found here

I also have a 5 year plan which outlines the "big picture" so that all stakeholders know where we're going year to year. That plan can be found here . Of course I watch for major industry trends and refine the 5 year plan in response to changes in technology, legislation and compliance requirements, but in general, the 5 year plan is a predictable roadmap of what we'll accomplish over the long term.

Many days, it feels like my job is driven by the contents of my Inbox, but in order to create a successful organization, I have to insulate my staff from attempts to change priorities on a daily basis. Instead, my job is to triage the entropy of my inbox into a few short term urgencies while protecting the operating plan and five year plan. At times, this triage exercise is challenging and I involve my various governance committees when decisions are politically charged or involving competing stakeholder priorites.

My advice to CIOs is to develop standard escalation processes and use those well defined processes each day in response to the tyranny of the urgent. Do not let your inbox dictate your strategy or priorities. Keep your eye on the big picture. As Jim Barksdale says, "The main thing is to make the main thing the main thing."

Friday, November 9, 2007

Blogging about Blogging

I've just completed my third week of daily blog entries.

This blog is truly an experiment about how a blog impacts communications with my staff, my vendors, and my colleagues. Thus far, it's been a great learning experience. I've found that I can answer many questions in public forums by referring to postings I've made in my blog. I've posed several questions to the community via my blog and received many helpful responses. The big question will be - it is sustainable? I typically sleep 4 hours a night. Filtering the events of each day to develop a coherent blog theme, writing the blog and responding to comments takes about 30 minutes every day from my 4 hour sleeping hours.

Thus far, blogging has not reduced email, phone calls, or the number of daily controversial issues I must address.

If I can share my experiences, good and bad, with the entire community and catalyze good things to happen, I will feel completely satisfied with the time investment.

People who know me well know that I am not driven by fame or fortune, I just want to make a difference. My experience running a software company in my early 20's taught me that judging the value of life by a number in a bank balance, the type of car you drive, or the size of your house is not very satisfying.

I also crave learning. I welcome suggestions on how to become a better blogger. The world of blogging, wikis, Facebook, IM, and Second Life is a new experience for me and I hope to synthesize all of these new media pilots into a coherent, time efficient framework for communicating with all my stakeholders. Wish me luck!

Thursday, November 8, 2007

The Dark Side of Going Public

Over the past 10 years in my CIO role, I've been customer and counselor to many technology companies, some public and some private. Although all companies are different and each has its own unique story to tell, there are a few trends I've seen in public technology companies that make me think private companies have a special appeal

1. Ronco sales tactics on the last day of the quarter - "buy our new gizmo today and we'll discount it 50% as well as include a set of free Ginzu knives". Pressures to meet the quarterly numbers and meet stock analyst expectations force salespeople go to extremes to close sales. As a CIO, I refuse to participate in such end of quarter blowout deals. My experience is that today's once in a lifetime deals will be there on the last day of next quarter. This kind of bizarre sales behavior mortgages the future the company by creating less profitable deals in the short term that impact the company in the long term, generally after the salesperson has left.

2. Obsession with stock options - employees want to be millionaires and options seem to offer the road to easy street. This forces the CEO and Human Resources to spend an incredible amount of time developing options policies, strategizing to retain valuable employees with options, and updating Boards/Comp Committees on options issues. Such efforts distract the CEO from focusing on the products and services of the company. Admittedly, options allow companies to attract top notch talent, but by setting expectations that millions can be made through options in a year or two, there is less interest by all in building an organization for the long term by product innovation and customer service

3. The crushing burden of Sarbanes Oxley - I've see public companies and companies about to go public spend millions on SOX compliance. Of course Enron and WorldCom were inexcusable examples of corporate greed. However, the impact on honest, growing new companies is so extreme that dollars need to be taken from R&D and applied to people and systems with no other purpose than meeting SOX requirements. With less R&D resources, companies innovate more slowly, to the detriment of all.

4. Focus on the shareholders/stock price - I've seen some very odd behaviors motivated by stock price. “Let's do a merger to create visibility and enhance the stock price.” “Let’s announce a new product before it is ready to generate buzz.” The end result is that marketing departments are a year ahead of engineering departments. Customer service is outsourced to enhance profitability rather than build long term successful relationships. Product quality is less important than mergers and acquisitions.

5. Challenging Board relations – As a leader in IT, I am evaluated by the quality of my service, the success of my projects, and the satisfaction of my employees. Wouldn’t it be great if the CEO could present product innovations, employee feedback and customer loyalty metrics to the Board and be rewarded for success, instead of being managed as a function of share price?


I know that it sounds retro to believe that CEOs and employees would be satisfied by the wage and bonuses generated by excellent products and services. That CEO salaries would be 25-50 times the average employee wage and clearly computed based on performance metrics rather than multimillion dollar options gains. That companies would be built to last decades and not the boom-bust cycle of growth, IPO and decline.

In my field, Healthcare IT, there are a few very strong private companies such as eClinicalWorks, Epic Systems and Meditech. These companies are product and customer driven. They have thousands of engineers and only dozens of salespeople. They gain market share over time because of their amazing responsiveness to customer needs. Their customers dictate their growth rate, their salaries, and their strategy.

Of course I realize that the public markets give companies the capital they need to expand and that many companies have large startup costs that require substantial external investment. However, as an IT customer, I often wish that my needs were the most important driver to the Board, CEO, and employees of the nation’s IT companies. My advice to new IT companies - consider staying private, your customers will thank you.

Wednesday, November 7, 2007

A Downtime Lesson Learned

Earlier this week, we had a few minutes of unexpected downtime in our mission critical clinical systems. The cause was interesting - a programmer accidentally requested our computers to do more than they were capable of doing. A runaway process brought a very powerful highly redundant cluster to the point of unresponsiveness. The cluster was so saturated that even our system administrators could not log on to kill the offending processes.

The event led me to examine all our last year's unplanned clinical system downtime. Three of our four unplanned events were related to runaway processes taking over computers before system administrators could intervene. Thus, we have highly redundant, highly scalable, change controlled infrastructure and software, but today's technology does not give computers the intelligence to say no when asked to perform processes that exceed the computer's capacity.

Over the next few weeks, we're going to think about automated watchdog systems that can take actionjust before the computers pass the point of no return. Actions could be killing runaway processes, reserving sufficient computer resources for high priority tasks such as system administration interventions, throttling processes that take more than a certain amount of total resources, or preventing new processes/sessions of a certain type from being initiated after a certain threshold is reached.

If anyone has any experience with building self-regulators into larger server farms, let me know!

Tuesday, November 6, 2007

Call me if you can

I thought my needs were simple: mobile e-mail and voice communications from one device that works in every country on the planet. One phone, one SIM card, voice and data everywhere. It's not as easy as it sounds.

Turning to Wikipedia, I found that the Global System for Mobile Communications (GSM) is a second-generation (2G) technology used by over 2 billion people in more than 212 countries and territories. It has four major bands. GSM-900 and GSM-1800 are used in Europe, the Middle East, Africa and most of Asia. GSM-850 and GSM-1900 are used in the United States, Canada, and many other countries in the Americas. It's the system of choice for AT&T/Cingular.

Code division multiple access (CDMA) is a 2G technology used by over 350 million people worldwide, primarily in the U.S., Canada and Asia. It's the system of choice for Verizon.

Universal Mobile Telecommunications System (UMTS) is a 3G technology used by over 100 million people in 48 countries, primarily in Western Europe and Asia, including Japan, where NTT DoCoMo uses WCDMA, the most popular form of UMTS.

Once I mastered the alphabet soup, I set out to find a voice and data device that provided the coverage I needed. Of course, most devices are locked to a specific carrier, so you need to shop for a device/carrier combination. My experience in the U.S. is that CDMA/Verizon provides great call quality everywhere. GSM/AT&T may work in Paris and Stockholm, but try to complete a call in Boston.

Time to check out the phones. How about a cool new iPhone? It's a quad-band GSM phone (850, 900, 1800, 1900 MHz), and AT&T is the only carrier. Per my comments above, this means great service in Europe and China, and dropped calls throughout the U.S., and no Japan. Close.

How about BlackBerry? The 8830 sounded great - full CDMA coverage by Verizon in the U.S. on the high-quality CDMA200/EVDO networks, plus full roaming on the 900/1800-MHz GSM networks in Europe and most of Asia. Pretty cool. No Japan, however.

Other BlackBerries, such as the 8800, are quad-band GSM just like the iPhone, with the same coverage issues. The 8320 offers quad-band GSM plus WiFi roaming for data. Unfortunately, few carriers support voice calls over Wi-Fi at this point.

But wait! There's an obscure device from BlackBerry called the 8707G. It's quad-band GSM plus 2100-MHz UMTS, so it works everywhere. In addition, it’s not tied to any carrier. You can feel free to use AT&T, Vodaphone, O2 or whatever local carrier is inexpensive in the country you're traveling in. Because it is unlocked, no carrier wants to sell it to you. You have to buy it directly from RIM.

And then there's the Palm Treo 750, with quad-band GSM plus UMTS. It's a Windows Mobile device sold by AT&T that comes with a little stylus to navigate the Windows Mobile interface.

I purchased both a BlackBerry 8707G and a Palm Treo 750. Both worked in the U.S. before I left for Japan. Both worked the moment I landed in Tokyo. Both worked in my European travels. The winner - the BlackBerry using AT&T service via a single SIM card that roams anywhere on the planet. Just try doing 600 e-mails a day with a little stylus on a 2-by-2-in. Windows Mobile screen.

I've come to the end of my quest. After weeks of struggle and hours on Wikipedia, I have a single device that works for voice and data on every square inch of the planet. Makes you wonder why the device manufacturers and carriers make this so difficult.

Sunday, November 4, 2007

The Unfriendly Skies

One of the side effects of being a Harvard faculty member and being chair of HITSP is traveling for teaching and collaboration around the world. In 2007, I've flown about 400,000 miles. This morning, on my way to give a keynote my flight was cancelled for no apparent reason. Every other flight to my destination was overbooked. The combination of high fuel prices, heightened security concerns, overbooked flights and surly airline employees makes flying a truly unpleasant experience. I finally asked the airline folk to consider other airports within 60 miles of Boston, then drove to Providence, went standby on several connecting flights and arrived on time to my keynote.

Days like today make me believe we should stop most travel to out of town meetings and shift to web-based collaboration tools. As I mentioned in my earlier blog entry , I'm testing these tools as part of an evaluation of flexible work arrangements. Today's experience motivated me to test video conferencing solutions.

I tested the following

Windows - Polycom PVX software, using H323 and SIP teleconferning protocols over IP.
MacIntosh - Xmeeting, an open source H323 and SIP teleconferencing tool
Ubuntu Linux - Ekiga, an open source H323 and SIP teleconferencing tool.


My first observation about video conferencing is that poor video can be tolerated, but audio must be nearly perfect for the technology to be useful. Polycom has figured that out, and seems to preferentially use available bandwidth to ensure the quality of the audio. I used the windows-based Polycom PVX software to connect via H323 to a MacIntosh running Xmeeting. It worked perfectly, offering 'good enough' video from my desktop Logitech Fusion camera and headset microphone. The MacIntosh side provided barely passable audio and passable video. iChat via IM seems to provide much higher quality audio and video on a Mac than the Xmeeting H323 approach. IP-based teleconferencing worked on these machines without any configuration hassales or configuration incompabilities. My experience with H320 ISDN teleconferencing which requires a series of digital telephone lines is that it can be quite finicky. Typically when I do ISDN teleconferencing, the engineers on both sides of the call need 30 minutes to ensure equipment compatbility and get the connection working. I've had many ISDN teleconferencing presentations fail completely, be interupted mid presentation and have variable quality during the course of the call.

My second experiment involved connecting a MacIntosh running Xmeeting with an Ubuntu Linux laptop running Ekiga. Although bandwidth should have been sufficient, I found that the Linux laptop did not perform the audio or video tasks well. This could have been because the laptop has low powered graphics hardware and only a 1.06 Ghz core solo, however, many other folks I've spoken with with have found that Linux does not seemed to be an optimal platform for high end real time audio/video applications at this time.

The bottom line of these experiments is that PolyCom seems to really have a business quality desktop teleconferencing solution that enables me to connect with collaborators using H323 protocols. Xmeeting came in second place, offering barely passive audio quality and passable video quality. Ekiga was not usable for business purposes, although it may suffice for casual chat.

The big question is whether or not the video is even necessary. Maybe a still photo and crisp bidirectional audio is sufficient for a meeting. The technology is ready to replace airline travel with desktop teleconferencing, but our business culture is not. We want to be about to reach out and touch the speaker, even if it means traveling thousands of miles and enduring the pain and expense of domestic airlines. As oil prices continue to rise and as climate change accelerates, it may be that economic and social forces will align to minimize physical travel and encourage us all to use low cost bandwidth from the comfort of our homes and offices instead.

The positive aspects of H323 was that the standards were mature, I did not encounter any firewall issues, and cross platform communication worked among all the computers and operating systems I own.

The downside is that it uses bandwidth for video that may not be truly necessary. My next step will be to explore the collaboration tools such as Webex, Adobe Connect (used to be Macromedia Breeze), and Elluminate to determine if audio plus shared presentations/whiteboards is a better fit to meet the needs of my road warrior travel schedule.

Friday, November 2, 2007

It's not a job, it's a lifestyle

I'm often asked, "What is your job?"

Some may think that being a CIO is all about bits and bytes, buying the latest technology and keeping up with all the three letter acroyms of the industry (I use WPA over EAP instead of WEP on my wireless network, what do you use?).

The technology portion of my job is about 15% of my time. The bulk of what I do is organizational, political, and customer relationship management. With hundreds of projects, thousands of customers, and millions of dollars, how do I keep it all straight?

My approach is 4 fold:

1. Strategy - The Information Technology department in every organization should not make the business strategy, that is up to the Board, CEO, and stakeholders. The IT organization should provide the tactics necessary to execute the organization's strategic plan. For example, if the strategy is to improve quality, the information technology organization can implement e-Prescribing, provider order entry, bar coded wrist bands, and incident tracking systems. Every year, I approach the strategic planning process by meeting with the CEO, CFO, COO, Chief Medical Officer, Chief Nursing Officer, and Chief Academic Officer to understand their strategic imperatives for the year. The next month, I meet with their direct reports to understand the operational implications and challenges of these strategic imperatives. I then produce an "IS Operating Plan" (note, that it is not a strategic plan), which is placed on the web for all stakeholders to review and presented to my IS Steering Committees. I have a Steering Committee for inpatient applications, outpatient applications, critical care applications, Operating Room/Anesthesia, Laboratory systems, Radiology systems, and Health Information Management (Medical Records). The chairs of each of these committees plus clinician representatives from the Medical Executive Committee serve on the enterprise wide Information Systems Steering Committee which ensures coordination of resources among all the projects. Once these committees approve the priorities for the year, we ensure the operating and capital budgets are aligned to do the work.

2. Structure - Once the operating plan is in place, I ensure the structure of the organization is arranged to support the projects to be done. Over the past 10 years I've done several mini-reorganizations to respond to changing technologies, customer needs, and governance issues. Note that the ideal structure is defined before taking into account the existing staff personalities and skillsets. To build an organization that delivers reliable service over time, I try to avoid single points of human failure, distributing work across many individuals rather than relying on a "lone genius", since reliance on one person is ultimately unsupportable.

3. Staffing - One the structure is in place, I ensure we have the best staff possible to populate that structure. I'm a strong believer in training and we try our best, given limited budgets, to hire talented people and continue their education so they remain world class experts. I'm also a great supporter of co-op programs for college students, bringing in new graduates, training them and hiring them to ensure a constant supply of new talent entering the organization.

4. Processes - Finally, once the staffing is in place, I work on the highly repeatable processes that support our workflow. The organization functions most smoothly when policies and procedures are well known by all internal and external to the organization so that I can monitor the performance of known processes, rapidly identifying areas where we can improve service delivery. Metrics I review include infrastructure uptime, electrical consumption, help desk call abandonment rates/time to problem resolution, budget performance by manager, and performance against project timeline milestones.

At the beginning of each day, I ask myself if the strategy, structure, staffing and processes are as good as they can be. At the end of each day, I mentally review the issues of the day affecting each of my direct reports and offer mid course corrections, which are most often organizational and political. I also try to communicate broadly via town meetings, email broadcasts, weekly leadership meetings, and monthly 1:1 meetings with each of my direct reports.

Finally, I try to reserve 50% of my time each day for the important issues of that day. Explaining to a customer or IS employee that I cannot respond to a critical issue for weeks because my calendar is booked far in advance does not work. If I can do today's work today, my calendar has the same number of meetings, but I become a real time responder to issues before they escalate. Having a Blackberry strapped to my belt 21 hours a day also helps me use my time efficiently and ensures I am not the rate limiting step in any decisionmaking process.

Of course I have a family life, personal time and outside interests, but being a CIO is a lifestyle, not a job. I'm connected to the strategy, structure, staffing and processes of the organization 24x7x365.

Thursday, November 1, 2007

Bar codes, RFID, and Patient Safety

Like most hospitals, Beth Israel Deaconess Medical Center is focused on maximizing patient safety, quality and value. Over the past few years, we've implemented Provider Order Entry, Guidelines/Care Plans, and Electronic Medical Records. However, a great puzzle remains - how do we positively identify our patients so that we are confident they are receiving the correct medications, have the correct blood samples sent to the lab and receive the right blood products during a transfusion? For example, with perfect identification, we can create an electronic medication administration record that documents that the right patient received the right medication from the right person at the right time.

To accomplish the goal of positive identification of our patients, our staff and our medications, we spent the past year investigating two major kinds of technology - bar codes and radio frequency identification (RFID). In different use cases, each technology has its pros and cons. Based on our early work, we have implemented these technologies in various production settings in the hospital with positive results.

Bar Codes have been used successfully in industry for decades. The technology is stable and well standardized. Bar codes come in two basic forms, simple linear bar codes and more complex two dimensional bar codes. Linear bar codes encode a few characters or numbers, such as the 12 digit UPC symbols found on grocer's shelves. Linear bar codes can be used in a healthcare setting to encode a medical record number placed on a patient wristband. Two dimensional bar codes can encode more complex information such as patient name, age, gender that could be used to provide details about the patient without requiring a lookup in a hospital information system, which helps provide redundancy in case of hospital system downtime.

Many vendors offer wrist bands and printers which facilitate easy bar coded wrist banding of each patient upon admission or registration. Bar codes are inexpensive, highly reliable and generally already used in hospitals. Hence existing hospital bar code readers can be used.< However, bar codes do have limitations. Reading bar codes requires line of sight scanning which means that patients need to be awakened/repositioned each time the bar code needs to be scanned. Linear bar codes must be relatively flat to be read properly, so a wristband wrapped around a premature baby's wrist is problematic. Bar codes generally do not read well when wrinkled, wet or torn. One positive aspect of this characteristic is that bar codes rarely yield an inaccurate read, but simply do not read at all when damaged.

In our case, we believe that putting both linear and two dimensional bar codes on wrist bands enables us to take advantage of linear bar code readers already in use in the hospital, while also preparing us for purchases of future bar code reading equipment which will read both two dimensional and linear bar codes. For employees, our security badging software produces a linear bar code on employee badges. For medications, an informal survey of our supply chain revealed that 70% of all medication containers are already bar coded. Although this may help with drug distribution, it does not help us identify unit doses of medications since pills are not yet bar coded. Repacking pills into bar coded bags is required for positive identification of unit dose medications.

Radio Frequency Identification is an evolving technology that is widely speculated to replace bar codes over the next few years. However, RFID in healthcare requires careful examination, because separating reality from hype can be challenging. RFID comes in two basic forms, active and passive.

Active RFID tags contain a battery and transmitter which can be used as a geo-location, constantly providing information about the physical location of the active tag in the hospital. Current active tags are about the size of the pager, require battery replacement every 6 months and cost $50 each. As with many new technologies, the size is decreasing, the battery life is lengthening and the cost per tag is dropping significantly. Active RFID transmitters generally use one of two frequencies - either 802.11b/WiFi (2.4 Ghz) or a proprietary frequency (488 Mhz). The advantage of using WiFi is that the existing hospital wireless network can be used to read tag location. Our experience is that Active RFID over WiFi can be rapidly and cost effectively deployed for use cases which require room level tag location. Proprietary systems that use an Active RFID specific network, such as 488 Mhz or infrared receivers in each room can provide location to the level of the square meter, but do require the installation of dedicated wiring to support the RFID system. >In our case, we believe that Active RFID tags are a robust technology for applications which can utilize a pager size device such as tracking equipment, tracking patient beds, and tracking staff who are willing to wear an extra pager-sized device. For tracking staff, we specifically worked with managers to ensure that tags would not be used in a punitive way i.e. to record minutes in the lunch room, trips outside to smoke etc. Our application of active RFID is currently for equipment tracking in the Emergency Department and has reduced the time to search for ventilators, IV pumps, and EKG devices to near zero.

Passive RFID tags contain an antenna and a chip, but no battery. They can be as flat as piece of paper and as small as a grain of rice. When a reader provides RF energy which is absorbed by the antenna, the chip is stimulated to broadcast its data. This data could be simple such as a medical record number or complex such as name/gender/date of birth. Examples of passive RFID tags are the Mobil Speedpass used for gasoline purchases, product identification tags used on retail products such as Gillette Razors at Walmart and tags used in libraries to track books.

RFID tags have several advantages over bar codes. They do not require line of sight reading, hence an RFID reader can be brought near a sleeping patient or a swaddled NICU baby and can easily read the patient identifier. RFID tags are resistant to moisture, crushing and tearing.

However, passive RFID is not a panacea. Tags are more expensive than simple printed bar codes. Standards for passive RFID are still in evolution and many different frequencies are used to read different tags i.e. 125 Khz, 134.2 Khz, and 13.56 Mhz. RFID tags typically have up to a 20% failure rate in manufacture and thus can result in a non-readable wrist band. RFID tags are much harder to read if a metal barrier such a aluminum foil exists between the reader and the tag.

Existing passive RFID products include wrist bands and implantable chips such as those used to track pets. Human use has been limited and I am one of the early evaluators of the technology (see Straight from the Shoulder in New England Journal of Medicine, July 28, 2005, page 331). My body is RFID enabled and when scanned, I emit my medical record identifiers which can be used by authorized physicians to retrieve my medical records via a secure web application.

In our case, we use Passive RFID to track NICU babies via RFID wristbands and to track mother's milk stored in tagged containers. A software application and RFID scanner is used to ensure the right infant receives the right milk and to automatically create an audit trail.

Our early work with positive patient identification can be summarized as

For identification of most patients, we believe linear and two dimensional bar codes on wrist bands is robust, cost effective and standardized. For staff badges, linear bar codes work well. For NICU babies passive RFID enables scanning of swaddled infants without disturbing them.

For identification of medications, we believe linear bar codes of NDC numbers on heat sealable plastic bags provides a practical means to positively identification medications.

For identification of equipment, specifically for tracking location in real time, active RFID works well. Because of the size and expense of tags, we do not believe active RFID should be used for patient identification at this time.

Thus, a combination of bar codes, passive RFID and active RFID is working well in our various pilots. No one technology meets the needs of all use cases. Although we favor bar codes over passive RFID in the short term, we do expect to eventually replace bar codes with RFID once the technology is more robust, standardized and cost effective.