So I’m basking the laurels of solving a performance issue, transforming a 2 day+ ETL process into a 15 minute process, when the next Big Thing comes down the pipes at work. I have to add five new columns to a feed that I created about five years back.
The feed itself was designed to use a newly implemented vendor system that we barely understood at the time. As our understanding of the system evolved, so did the feed. Two years and 46 scope changes later (all of which I worked on), we had working-yet-kludgy ETL process. It did what it needed to do, but wasn’t pretty.
Because the feed was based on scenarios instead of business rules, there are a lot of “if Skeeve is wearing yellow and it’s Thursday, then send this information, but only if the zeroes are negative and only if we didn’t forget the cheddar on the turkey sandwich. And if the sandwich is chicken, then send this other data, but only on Fridays” type of coding.
I eventually got around a lot of these scenario “rules” by creating data tables which drive the code instead of hard coding issues. But the feed is still a delicate and dicey thing that requires delicate handling whenever new changes are made. I know this. I’m the sole dev on this feed. No one else has ever touched it. So I should remember to check all the old stuff when I add something new.
My problem was that I got this last project update at the last minute (3 business days before it was due into Test). Yeah… Excuses, excuses. I powered through the weekend doing 60+ hours of work, and felt very pleased with myself for getting everything working perfectly by the test deadline. Except, well…
The day after testing started, the requirements changed. (I challenge any DBA or Dev to show me an environment where that doesn’t ever happen.) As I double-checked the changes, I found a small notation I’d missed. Whoops! So I head back to coding, fix the issue (which is fortunately minor), and get the changes sent to Test. Then today, the tester and BA come back to me and say, “Um, you missed this other thing.”
So I’m supposed to add five columns which send records based on scenario A and scenario C. But the problem is that half the new records that are supposed to send in scenario A ran into an old rule that says none of those records are to be sent in scenario A.
And now the pain truly begins as a new rule contradicts an older rule that I completely forgot about. #headdesk
It can be fixed. I will fix it. In the meantime, I will be banging my head against the desk and reminding myself that even for a veteran DBA, SQL is hard. Not so much the basic administration or basic coding, but the job itself. Just when I think I’ve got things handled, a curve ball comes up and smacks me upside the head.
And yet, that’s why I love being a DBA. Because despite the repetitive tasks, at the end of the day SQL is challenging, new, and different. There’s always something to learn and the complexity, while annoying at times, is fun.
Is SQL hard for you? Or am I alone in this opinion?
