Published on August 29th, 2024
I'd like to share a recent experience that highlights the importance of refactoring in the face of performance issues. I was working on a part of my company's customer application that was responsible for loading a substantial amount of data. In this project, I was converting and re-working our legacy login management system, updating our API endpoints and modernizing the frontend interface. As the volume of data grew, the page load time began to suffer significantly, often taking over 15 seconds to load. On one test account, with over 1000 logins, the time to load the page could exceed a full minute. After working through our legacy endpoints with a fine-toothed comb, I managed to drastically reduce the load time and enhance the overall performance by writing a brand-new SQL procedure.
Previously, our API endpoints were referencing legacy SQL procedures, written almost a decade ago. The output from those procs were being loaded using outdated classes; specifically, these classes were designed to be bandwidth-saving at the cost of performance. However, as the system evolved and the amount of customer-related information expanded, the existing methods started to become a bottleneck. This procedure and these dataloader methods, which were tasked with retrieving and filtering a large set of login data, were slowing down the page load time considerably. Despite several attempts to optimize other areas, the improvements were marginal, indicating that a significant refactoring was in order.
To address this performance issue, I embarked on a refactoring journey.
First, I created a new, more efficient SQL procedure.
We only retrieved exactly the information that we needed to process these logins - no more, no less.
Next, I used a modernized dataloader to safely and quickly process the procedure output.
And finally, I implemented this into our API endpoints.
The impact of this refactoring was immediately noticeable, to a degree that was genuinely shocking.
Page load time was drastically reduced; my personal test account, which had previously taken anywhere from 2-6 seconds to load the page, never exceed 0.35 seconds now.
Unfortunately, I cannot share the exact SQL script or sample videos, but it was a significant improvement and a great capstone for this project.
I'm sharing this to emphasize the important of revisiting and refactoring legacy code to improve performance.
As our systems evolve and our data grows, our code should adapt accordingly.
Methods that were once effective may become bottlenecks as the volume of data increases.
Refactoring, while sometimes a significant undertaking, is a necessary process to ensure our software remains optimized and efficient.
People might argue and tell you that the old method works fine, so you should find other places to improve performance.
They might tell you to batch the return data, loading sections at a time, or they might tell you to reduce the number of for-loops.
I think this is all true, and these suggestions have their place in improving page performance.
But in this particular case, and in other future cases, we should have the strength to stand our ground and fight for a full refactoring when needed.
Did writing this SQL script and adding dataloader support take a while?
Yes.
Did I have to argue my case, and convince my team members that this was the right path forward?
Yes.
Was it worth it for improving the customer's experience?
Absolutely.