However there always needs to be a plan on how to migrate data that uses the old way to the new way. It’s not as sexy, but it needs to be done and takes time away from building new features. Skipping the design process is tempting, but it’s a direct path to this type of technical debt that’s easy to foresee.
As a software dev my gut feeling is that this requires the polar opposite of move fast and break things — instead of going for the obvious solution it requires you to think a bit more how data flows (or ought to) flow through the organizations using the software.
You can’t anticipate all design changes, but you can get better at making them so they happen faster. This means getting practice at doing them.
If you're serious about writing robust and reliable systems, think about V-1 and V+1 not just the version you are writing and deploying. The point about your code interacting with multiple versions in production is a good one, but also upgrades and downgrades should be considered.
How did you miss something so close to the beginning of the article? I’d love to hear how you “restore from backup” all the external systems that data propagates to.
Tbh the article's actual example is stupid, if I desperately need to rollback that change I'd just take a backup of that table and change the new types back to the main payment method, 'Stripe'. When we've got it fixed and redeploy the changes, change them back.
Problem solved in two SQL queries.
Not sure if the author's never had to deal with a problem like this before, but his 'impossible' problem's got a bloody obvious solution.
SELECT * INTO _paymentsbackup FROM payments WHERE PaymentType in (3, 4)
UPDATE payments SET PayemntType = 1 WHERE PaymentType in (3, 4)Thank you!
If business domains are properly abstracted, most of the time extending the software with changes is something close to adding some configuration or using existing functions with different parameters. But the hesitation to create things from scratch comes at the cost of training and also transparency, two of the highest priorities for managers and stakeholders who not only want to hire people with minimal overhead to begin contributing, but also want to understand as much as possible what the engineer is doing and how they are doing it.
Software is incredibly maleable and has unlimited potential, yet somehow most places i've worked at get constantly bottle-necked by recurring problems that are very easy to solve with proper abstractions.
Even knowing that and attempting to build something with best principles is still going to be a pain to maintain as it's always a moving target with no proper specs.
It can become extremely difficult to figure out if you mar yourself with someone else's framework. That is quite true. But is also what the parent said.
Domain Driven Design basically aimed at tackling this issue, by trying to synchronize abstractions across departments, so I get that there are no easy solutions, especially in large orgs.
It's way easier to tell your developers to sprint every two weeks in some random direction, because at least then the business has control. If they were to use the scientific method, the direction would be chosen based on experiments and empirical data that they don't understand or control. Can't have that.
That's not necessarily bad. The company is derisking by relying on building blocks developed by established companies with deep talent. The vast majority of projects where local developers wrote everything from scratch are nightmarish to maintain, secure and upgrade.
In general I agree with you, but the devil is in the details and a licensure body is surely the wrong way to do this.
In most so called "engineering" orgs, the scientific method is explicitly banned - decisions are made by weird scrum rituals and fabricated storypoint metrics.
Which fields do you think are allowed to do "proper" ________ in a typical business?
If you give me the choice between working with 2 groups of people where one says, "Let's get this done the right way!" and one that says, "Let's get this done!", I'll join the second one every time.
And I'm not saying it's perfect, I'm just saying it's done properly.
But at the same time I’d bet if you asked people in those professions they’d likely say, “Our bosses don’t let us do things the proper way.”
It’s more likely everybody thinks they know better than their boss regardless of the field.
But that doesn't mean you throw the baby with the bathwater. I was still liable for things like fire hazard or RF interference, among other things.
What other fields do you believe are not heavily constrained and are allowed to exercise their full expertise at a typical business?
My point is that nearly everybody feels constrained at work because of decisions from higher-ups or whatever. It’s not like software developers are some unique set of people who face challenges nobody else faces.
The problem is that it's still hard for software engineers to go to business people and say "no, we can't build a bridge that crosses the Atlantic Ocean".
With the introduction of non-technical Product Managers it became even harder. Now we have to convince two layers of people that some idea is unsafe or that it's gonna be more trouble than it's worth.
It speaks again to the lack of rigor in software. In professional engineering practice, the project manager is always a P.E. who takes responsibility for the final design and signs off on the product before it goes out the door.
PE licensure could also foster a security culture in software like none other, as no PE is going to put his fortunes on the line to save a buck skipping formal verifications and avoiding inconvenient best practices!
OTOH there are many genuinely great products that don't require massive upfront effort to start producing value. Demanding tailored demos can help you separate the wheat from the chaff.
You would almost certainly see an increase of in-house software developers and consultants where the code is never charged for, but is developed as it always has been. A few big companies like Microsoft and IBM would get certified and smaller software companies would be driven out of existence entirely.
There would definitely not be a golden age of rigorous software.
App keeps updating. Version N understands version N-1's saved data, maybe N-2, N-3, but not indefinitely. Still you're lulled into a sense of security; your data will always be retrievable. Goes without saying that the app modifies the data store to the latest version.
But then you don't use it for a while and you it's now at N+10 and you still have data from N, and it's slid out of the "backwards compatibility window" and is no longer readable. And the intermediate versions of the App are not available. Now you're screwed. One such scenario, for sure, is updating from version N to, say, N+5 of Android and trying to migrate your data, but I think it was something else.
This is where good abstraction, while expensive to develop, is important: if the abstractions are well-formed then fixing code or adding features should not cause downstream breakage. We always want downstream to focus on using our projects to make cool shit, we don't want downstream spending most of his time fixing his abstractions because we changed ours for the fifth time in seven years.
Version 10 should have the figurative similarity matrices to project any version of the format from v1 to v9 into v10 format.
And I mean, I could quibble, but let's roll with it.
Good luck getting finance to work with "event store-based spreadsheets", finding an "event store data lake" for your analysts, getting that other engineering team to also switch to an "event store"-based view of the world when they just want to query the system about a point-in-time about whether a user is up-to-date and aren't going to make another one for another month no matter what you do to your data store, so you'd better be right....
Event stores are an internal implementation detail, like whether or not you use Postgres or MySQL. They don't save you from the effects of changes and the parts of your state that leave your control. The example is well-chosen... billing state is going to leave your system, and the business forces causing it to leave your system are going to be powerful to the point that if you try to stand in front of them and yell "STOP! You all need to use an event store architecture if you want billing data!" that you will be fired and replaced.
Which is not to say that it's all hopeless. You can take care of the data well inside of your own domain. But as I know from experience, you're not freed from these forces just because you've got a good architecture. You really, really want to careful how you proceed forward, the moreso because nobody else working with billing data is going to be.
I'm responsible for the "usage based billing" portion of the billing system. I've often thought my primary contribution to the billing system hasn't been the usage based billing, but sitting in on the meetings as a senior engineer with a bunch of other people who are not engineers and whose programming teams are also busy with other non-engineering tasks, asking the right questions, and making sure that we get down to the true crux of the matter and find the exact right groups to do the right tasks with the right data coming from the right place, and making sure we don't get a whole bunch of workarounds from the wrong people sending the wrong data to do the wrong tasks, and then someone somewhere "just" "fixing" it later. Even when, and perhaps even especially when, the final task results in no action items for me or my team and all I did was ask the right questions, it is just so much easier when you don't make architectural mistakes in the first place in these cross-team decisions. Basically, using those requirements elicitation skills more than anything else.
My system basically is an event store. That doesn't so much solve the problems on its own, as keeps me flexible and able to respond to the other much less flexible systems as we work through problems.
> aside: This email is adapted from a book chapter I spent the last 3 weeks writing. It's a complex idea that took 10 years to grok. I tried squeezing it into a digestible length. Would love to hear your thoughts.
Okay...
Next line:
> Data lives forever
It doesn't. Now I'm unwilling to overlook the title, which is clickbait, so I'm done reading.