Paying down technical debt

One of the cool things that I do as an Evangelist at Redgate is to periodically visit company headquarters in Cambridge. The other Evangelists and I get to meet with every software developer, product manager, and UX designer at Redgate over a series of meetings. That’s really cool. We talk about things that they’ve released lately, what they’re looking at doing in the near future, and we get to give feedback based on what we hear from the community and from folks in the sales process. We also get to share what we personally think should happen in these products now.

As you might imagine, I have a wish list for features in a variety of different Redgate products

Our products are great, and one of the things about great products is that users are always inspired to want to use them in new ways, so I never lack for ideas.

So, I have a lot of opinions about things that I think should happen, and features that I would love to have for customers. And, of course, I’d like those features right now, please.

In a recent meetings with one of the teams, they mentioned that most of their work over the next couple of sprints involves working on their continuous integration process

This sounds like a bummer, right? It’s a time period when the features on my wishlist, and Steve‘s wishlist, and Grant and Kathi‘s wish lists aren’t getting worked on.

But what the team explained was that this application has been around for a while, and a large amount of tests have accumulated. It currently takes more than 10 hours for the build and test process to run. There may be duplicate work going on in the tests, and there are probably tasks that can be made much more efficient. The long build and test time currently makes it painfully slow for the team to iterate on developing and testing new features. 

The team said that in the long run it’s worth paying down some debt and making the automated build and testing cycle more efficient, so that they can iterate on features faster in the future, instead of having to find other things to do while waiting for the CI process to complete.

This news wasn’t greeted with cheers from all the Evangelists present — but, to be fair, when we do respond to something with cheers it makes some of the teams look at us oddly, as we’re in the UK and that’s not something they see every day at work. (Hey, I bring my American enthusiasm everywhere!)

But everyone in the room agreed that speeding up the build and test cycle as much as possible is a necessary and reasonable thing to do

Like any other set of users, we want what we want (and we want it ASAP), but we respect that to make software development work well, you occasionally have to step back and pay down some technical debt.

This is also true for database development

It’s not always obvious that when doing DevOps, stability is just as important as release frequency, but that is the case, and maintaining that stability requires being diligent about tidying up one’s processes.

As database professionals, as developers and DBAs, I believe what we should see as the ideal release cycle is one in which we are free to release features every day without any manual work, and we have a software development cycle in place that ensures that the risk of our changes is minimized, that coding patterns are being used that ensure system stability, and that we have a response process in place that restores services as quickly as needed should there be a performance, availability, or functionality problem.

But that doesn’t mean that we actually release changes every single day. In order to do that effectively, we usually have to have paid down a lot of technical debt. That means stepping back periodically and working on improving our processes, rather than relentlessly focusing on shipping, shipping, shipping, shipping.

This is even more critical with legacy applications, where there is a significant amount of technical debt to pay down

Spending release cycles on making continuous integration and continuous delivery/deployment work better isn’t the part of DevOps that gets business owners and users really excited. But it’s still important to talk about, because this is a critical activity that enables us to deliver value on a regular basis — and that is what gets those folks excited.

4 Comments. Leave new

  • Derek Bell
    May 8, 2019 6:47 am

    I find that technical debt in the context of database development/management is problematic, especially when the data model was poorly implemented (i.e. lack of understanding relational concepts, change tracking, slowly changing dimensions, indexing, table types, etc.).

    I have witnessed several cases where the data modeling was poorly executed, making it that much more difficult when the application developers wanted to implement new features as well as when the total storage grew beyond what the model could handle. I have argued with developers and business analysts that, “No, you shouldn’t connect directly to tables because I reserve the right to change them.” Views and store procedures can be updated so that any logical design changes won’t adversely impact their applications or reporting environments. So far, though, I have only “won” this argument when I was the original data architect – Never allowing the opportunity to present itself in the first place.

  • henrikstaunpoulsen
    May 13, 2019 6:57 am

    I’ve found that development teams can spent an enormous amount of time improving the “continuous integration process” or some other stuff, but that they often forget to measure the customer value of this. It is also difficult to define “finished”. May I suggest that the Evangelists try to define “done”, and that you check if better hardware might be a solution?
    Yes, I’m a developer myself.

    • In the case of our team, I recall they mentioned that for years they’ve been throwing hardware at the problem as the way to try to keep it under control fast, and it’s not tenable anymore. We do also value autonomy for our dev teams and while the evangelists have influence and we have lots of collaboration, we don’t define “done” for the dev teams, and we do trust them to do that.

      • henrikstaunpoulsen
        May 16, 2019 12:17 am

        yes, that is a better idea; Let the dev. team define done. I would still want to monitor and measure “done-level”, as I think we (the dev. team) easily forget the cost, and let a job go on and on.


Share a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: