This interview with Todd Simpson, the Deputy Assistant Secretary for DevSecOps at Veterans Affairs (VA), discusses DevSecOps at VA, EHRM, the concept of Technical Debt and what’s ahead for FY21.
Behind the Scenes at VA
When I came to VA, the first thing that was apparent was the calibre of the people within the organization and how committed, skilled and hard-working the team is. This place moves from dusk to dawn like a lightening bolt, fast-paced and energized. My team moves from planning sessions to meetings, solving challenges, working with business customers and end-users, taking in requirements. Everyone is so committed to the success of the mission.
I also quickly realized the overall maturity. I salute Mr. Gfrerer every day when I find the little things that show he has this organization pointed in the right direction. It was clear they had already started along the DevSecOps journey before I arrived by the tools that were in place, the maturity of the processes, and the goals that were already in place around measurable outcomes. The team was already positioned for success.
Often over the course of my career I have been introduced to teams where there were structures that needed to be changed or a culture that needed a reset, but all of that was in place and working already here. There is still work to be done and there are always places to improve but the people, process, and technology components are already positioned for successful continued transformation.
DevSecOps at VA
Foundationally, DevSecOps requires a robust cloud environment or ecosystem. In agencies that may not yet have that positioning, that is where the work needs to start. At VA we already have that, we have established relationships with partners and a fairly robust and mature cloud ecosystem so we are in a good place. We will need containers and for those to live somewhere, but we already have the garden in place so to speak where those can be planted.
Within VA there is already the cultural transformation in progress that is required for success, and people have already started embracing the way DevSecOps needs to move forward. That started here in 2018 when the Department moved to product line management and to the cloud. VA has made a serious dent in its cloud application footprint in a short period of time, with 95 applications and systems already moved to the cloud.
We are all focused on working toward business outcomes and working with partners to build pathways to intake new projects, focused on ensuring we have good governance and intake models. Being able to reprioritize initiatives, to drive toward business outcomes, will ensure we can continue to move forward.
Within the organization we have key leadership within the team identified and empowered. We have begun to bring our development, operations and security organizations together to ensure they are more closely aligned, to ensure there are leaders in place to run the platform groups.
That was one lesson I took from my time with the Department of Health and Human Services (HHS). Having a platform manager and a platform director, allowed me more efficient product line management. It is a similar process for anyone who is moving from complexity to commodity.
We are also working with peers to eliminate incompatibility in processes that may be holding us back, looking at vendor processes from an agile view to ensure we are not bottlenecking around old processes. And we are standardizing approaches to incident management and problem identification.
From a tool standpoint we are making sure we have the appropriate tool chain in place to support not only the creation of stacks but that we are baking security into our models, and blowing away old environments when they are not needed. There was good work going on before and we’ll reuse as much in our toolset as we can, but we will certainly be adding to it.
DevSecOps and EHRM
The endgame for Electronic Health Record Modernization (EHRM) is a unified Veteran experience. There are a lot of lessons we can learn from many of the major projects we’ve tackled the past few years that will help us deliver smarter for EHRM. There are examples throughout the Department of DevSecOps practices, agile, and user-centered design. VA has a track record of success to reference and has made significant movement applying these kinds of methodologies to deliver business impact to customers.
With respect to EHRM, we view it from the same lens as other projects where we are focused on vetting methodologies, enhanced collaboration, ensuring best practices and working with the common goal of doing all we can to meet the challenge of that overall digital transformation.
Another project we have been working on is transforming the memorial system to deliver better value to the customer at what is arguably one of the worst times in their lives, trying to make the experience better for the loved ones left behind. It all comes down to user-centered design, working with the Agile Center of Excellence on processes and maturing the steps in the delivery model.
We are developing scorecards that are standard around certain metrics like lead time for change and deployment frequency, then doubling down on the key metrics to move from one maturity level to the next.
During COVID, that approach allowed us to develop 30 new SaaS products in a very short time and our metrics tracking allowed us to react to early warning signs and to stay on track with several COVID related efforts.
The Challenge of Technical Debt
Technical debt is a problem I have dealt with throughout my career but for the first time, I have this language and a way to quantify the problem under this term. Technical debt accumulates as hardware and software reaches its end of life. Things break and they are less secure and harder to fix. Technical debt accrues and it is a problem that won’t be solved overnight.
In the private sector, the way they deal with it is to have a refresh baked into every business case by year five. It has to be baked into that initial plan and investment. For us, it means asking customers to come to the table with well-defined capabilities and expectations, with an understanding of what projects will look like, right through the end of their lifecycle.
Within VA we have set a strategy in place to quantify our resources and to track the management of our total technical debt to achieve a balanced state in which we will service our debt in the same year it is accrued. That plan should see us in a good place within four years.
I know from experience that when we get to the point that technical debt has been addressed, there is an ability to make software changes six times faster, systems are more reliable and there will be more funding for new ideas because it is not being redirected to sustainment. Without such a plan it becomes paradoxical because you can’t modernize and are constantly dumping funds into sustainment.
A lot of the technical debt we have accrued has been the result of working to continue to provide services to a growing population with a budget that has remained the same and a human capital that has not grown in relation to the growing user population. Sometimes that just happens, but as we work to eliminate that debt, it will make our security posture stronger and it will better allow us to keep pace with our business goals.
Opportunities in FY21
Looking ahead, part of our focus will be on cohesion within the organization, rather than massive reorganization. DevSecOps is a methodology that delivers secure development solutions through automated processes. Within VA, DevSecOps is both the way we deliver solutions, and it is an organization. We are working to get the DevSecOps organization all under one umbrella with the business services, strategy, and product monitoring and support in place to ensure our product lines and support functions are all working cohesively across the organization.
We are maturing our product line methodology and support functions to bring more depth to our current projects and to enable us to understand their ecosystem. Whether its unfunded requests or signs that customer requirements may be slipping, we want visibility and the ability to surge resources in real time.
We are also working with peers around strategic sourcing to ensure we have governance over the tools at our disposal and the option to look at alternatives to be able to give customers choices in price and functionality. We’ll also look to as much SaaS as possible, moving from the complexity of legacy apps to more sustainable solutions. The DevSecOps framework we’re embracing will help us leverage solutions across the spectrum and deliver better outcomes for our business customers and Veterans.
About Todd Simpson
Mr. Simpson has 30 years of information technology (IT) leadership and management experience from the private sector and the Federal Government. For the last 15 years, he has led at the executive level focusing on strategic planning, technology modernization, organizational development and operational efficiency. Mr. Simpson began his career in the United States Air Force serving 3 years active duty and 3 years active reserve. He then worked in the IT field as a systems analyst, developer, network engineer, IT manager and CIO in the private sector for 18 years.
In 2010, Mr. Simpson began his Federal Government career as the CIO for the United States Department of Justice Criminal Division in Washington, D.C. In 2014, Mr. Simpson was appointed to the Senior Executive Service (SES) as the Associate Chief Information Officer at the U.S. Department of Transportation. From 2015 to 2018, Mr. Simpson was the Chief Information Officer (CIO) at the Food and Drug Administration (FDA) in Washington, DC. In 2018, Mr. Simpson was selected to be the first Chief Product Officer at the Department of Health and Human Services. In this role, Mr. Simpson was responsible for providing the Department’s data, cloud, platform, security and development services.
