I was hired 6 months ago as a freelance data engineer, and after proving myself through my work quality, I am now essentially functioning as a tech lead, with full responsibility and ownership of designing, implementing, and hiring for the projects I'm assigned.
Our company is not a tech company, so I only have a couple of tech-oriented colleagues, and I barely interact with them. Now I directly report to the director of the company, who in all senses is awesome, with 40+ years of combined experience in some of the biggest oil and drilling companies globally.
However, I have some strong FOMO about not being able to learn much technical stuff from my peers or seniors. I am trying my best to learn and pick things up on my own, learning design principles, getting code reviews from chatGPT, etc. But even then, I'm afraid I am not producing the software to the highest standards of the industry since we don't have any rigorous cross-checking, and might be missing out on a lot of learning.
Can someone who has been in positions similar to these please guide me?
If technical proficiency was all that important to your employer, you probably wouldn't be in position you're in.
This 100%. It seems that you're doing things proficiently enough to keep your job, but just want to improve your capabilities as a coder. It's going to be difficult without a proper structure or mentors around you. So you've to make a decision whether you'd prefer to stick with the role and get slowly better over time using blog posts and AI reviews, or jump in the deep end and get another role in a more tech industry aligned company. And if you decide to jump to another role, just be aware that you're going to spend 3 to 6 months panicking about whether you're in over you head, and probably won't get "comfortable" in the role for a year.
YMMV, though. Some data engineers are writing basic SQL or playing with Azure Data Factory and there isn't too much complexity. Read Designing Data Intensive Applications. If that sort of thing resonates with you then find someone to work with who has experience with those kinds of problems!
[Edit: no disrespect to ADF! Just pointing out that the data engineering discipline is broad and different practitioners will have different expectations of complexity]
Perhaps they feel equally isolated.
Where in the world are you? That determines your options.
Read and study and learn from your mistakes and the mistakes of others – you will believe things and realize later that your beliefs were wrong.
If there was one book I had to recommend that I have read it would be Sandi Metz's 99 Bottles series.
Oh, and learn multiple programming languages.
Let go of FOMO, youth is the only reason you think it is possible to not miss out. You have and will miss most things because they were not hit your way. The only things you can actually miss are the opportunities that are yours.
But the only way to miss them is because your attention was somewhere else. You have an awesome opportunity for a mentor in the director. You caught a lucky bus, stay on it.
Right now people can look at you and see potential. In just a few years, they will look at you and see squandered potential. Sure technically you are an adult, but you have almost zero adult experience (your mentor has forty years of adult experience). Good luck.
However, if you learn well by doing, or by reading, there are loads of other great ways to improve technically. I’ve made big leaps forward in my skills by building (relatively large) side projects, where I can safely experiment with different design decisions and see the consequences over time. I’ve also got a huge amount out of just sitting down and reading the docs for tech I’m interested in - some frameworks (like React) have fantastic resources that can take you from good to great.
Good luck!
Ask them what the original goal was, how that changed, what proved to be important, what was unexpected.
Find some software that helps your company, and don’t be afraid to dive into the codebase. There may be times you can even contribute as part of your job.
Open source projects maintainers very likely have their own day jobs that are unrelated or only marginally related to the projects, have different priorities or just don't care enough. A previously important feature that has seen lots of activity may be on the shelf, but that's only known among maintainers with no public notes. You don't get to go into meetings or just send a Teams/slack message to get a quick response, let alone a casual chat at the coffee machine.
Unless you are working on prioritized features on a project like linux, VSCode or Chromium, chances are that your issues or pull requests go in the log until someone works on it months or even years later.
Speaking from real experience.
I am self taught and had no mentors for the first decade of my career. During that time I did things nobody else could do, things I was told were impossible.
Eventually I read some books and they added useful tools.
Read all the books from SICP to Superintelligence (ie concrete to speculative fiction).
But nothing is as educational as putting what you read into production, or putting it into production and fucking it up. I’d rather work with someone who has deleted a production database (ideally a preproduction database to be fair) than someone who hasn’t.
1. Learn how to learn well, continuously, and sustainably. Tech changes rapidly. And you will want to hop from one domain to another, just for keeping things interesting and to move with markets. This is both a blessing and a curse. It is a blessing because you can start late and still be in the top percentile if you have the brains and work hard for it. It is a curse because you will be doing this no matter how many years of experience you have.
2. Hone your non-technical skills– caution: these are compounding over time (both good and bad habits) – being disciplined, thinking clearly, articulating clearly, being professional, being trustworthy, managing your physical and mental health, being dependable/reliable, having a growth mindset, thriving in ambiguity and uncertainty etc. then, honing your communication skills – effectively collaborating with people, give/receive effective feedback, do/get mentoring/coaching, working with cross-functional people, working with very seniors, very juniors, peers etc. read a lot, develop mental models, deeply craft your personal approach to first principles problem solving, to making tradeoffs/bets etc.
You can do the above all by yourself, through reading, and observing people from afar, and engaging with people (even strangers on forum like this one) in dialog.
To make that more actionable... My approach in life has generally been to find a project (even something seemingly incredibly dumb, as long as it is fun), then work through it, learning what I need to know as I go along. To learn "well", you must then also constantly question what you have done as you complete various stages of the project to see if you have done them as effectively as possible, and try to incorporate any lessons learned into future projects.
I have found that how individuals do the learning required for this differs significantly from person to person, so it is hard to recommend any particular approach.
For example, I discovered early on that I learn in three phases: 1. I get exposed to something (a concept, a process, etc); basically discover that something exists. 2. I then see how that thing is used whether through mentorship or tutorials or, increasingly, through trial and error. 3. I apply that thing to some novel problem.
Through this cycle of Discovery-Tutelage-Application, I can assess my level of comfort with new material and understand when my struggles are due to trying to short circuit the process.
It's likely that you have some form of learning process that is equally cyclical, yet undefined -- once you identify and codify those steps, you can evaluate your progress when it comes to acquiring new skills.
Then in a regular work I explicitly detect where it pays off and feel “see I told you”. This creates a motivational loop to continue not-adhd-ing through tech.
Sometimes I still fly over the knowledge, but then may note that what I’ve been doing in a complex way could be solved with one parameter, if only I knew about it. This creates negative feedback against flying over.
This is ofc only one facet of learning, but I find this “see I told you” method very effective, cause my main issue with learning is unwillingness to learn for no clear reason.
Now, I mostly actually enjoy doing this and thus it has not really limited me. But I wish I could just spend some actual work time on more ‘non adhd-ing’ what I learn.
Sadly I'm now so easily burnout that even setting a dev env up can burn me out.
I am not sure there's any hand holding that can be given to someone to learn how to learn.
In code you don't have all the extra communication avenues that we have when speaking, like body language, intonation, sarcasm and so on.
On the other hand, when writing code we are not speaking in real time. We can think about a problem for a while, consider the best possible way to solve a problem and how to explain it.
What do you see as a big difference?
(3) Communicating clearly is an orthogonal skill to coding clearly. I think this skill is barely acknowledged in engineering cultures in comparison to the above.
I feel you have to have an engineering culture that values institutional knowledge retention, team education and growth — and not treating engineers as fungible — to get to level (2). Level (3) would be a great place to work.
There's keeping the engine well maintained (the sleep and exercise part, for example) and there's driving the engine down new paths and honing you driving technique. You work on the latter by exposing yourself to novel and interesting arguments (interesting philosophy or argumentative non-fiction, for example) and then working through the argument again with counter-arguments in mind. I would not recommend pop-sci books for this, because their arguments and writing tends to be quite flabby.
I'd actually recommend something like RG Collingwood's "The Principles of Art" which is a relatively plain English example of well written philosophy:
https://archive.org/details/in.ernet.dli.2015.188470/page/n1...
You will disagree with him. The point is to understand how and why you disagree with him and to explain that clearly, not only to yourself but others as well. Thinking clearly is about responding and communicating clearly while displaying a sure and succinct understanding of the problem at hand.
You can apply that to everything you read or are confronted with, but the key thing to realize is that "thinking clearly" is something you practice. There is no one trick --- it's an approach.
Practical tips on learn good techniques, do research, find the best tech companies that do similar development to yours. Check out their technical blogs, their githubs, find opensource projects which have been developed to the highest standard. Dig into them and potentially even rewrite your own simple versions to learn, maybe twin it so you could make the new implementation a part of an internal research project... so main possibilities there.
It is so valuable to take the ideas and questions in your head and get them out. You can then have Claude interrogate and problem solve with you.
By default, I think Claude is very conservative and sometimes comes up with ideas that aren't quite right. But you can actually just tell it to be less concerned about X or Y, to start over from scratch, etc.
Another trick I use is to ask it to ask me one question at a time to help clarify my thinking. I find that the one by one questioning really helps me find the boundaries of my knowledge and crystallize my opinions.
What I did:
* Read. All the things. Read HN comments, read tech blogs, read books. Don't put too much weight on any single book/comment/blog post—they'll almost all disagree with each other, and you'll learn the most by comparing and contrasting all of the strong opinions rather than uncritically absorbing any given opinion.
* Be okay with using work time for professional development. Your employer hired someone they knew wasn't experienced into a role that is typically filled by someone more senior. They need to be okay with you learning on the job, and if they're not you need to go somewhere else.
What I wish I'd done but failed to do:
* Don't get overly invested and burn out. This is your first job, but it's not your life, and your career doesn't depend on you completely blowing it out of the water as a brand new developer thrown into the deep end. Don't overinvest.
* Keep your eye out for red flags. Tiny companies that can't hire seniors might be small and niche but sustainable, but they might also stay small because the business side of things just doesn't work for various reasons. In my own case, it turned out the founders of the tiny company were both narcissists (which seems to be a common problem with tiny companies), which led to a ridiculous political implosion that I was lucky to escape.
Join Discords related to the stuff you’re working on, you’ll find people much smarter than you and any of your colleagues hanging out, talking about good approaches to designs, structures, infrastructure, etc.
You’ll also find idiots not worth listening to.
Remember you’re 6 months in / just doing the job is enough of a challenge. You’re calling yourself a lead for having full responsibility of the projects you’re working on. This is typically common everywhere except large tech companies.
You can learn from everyone around you, regardless of their status. There is no "universal developer experience curve", everyone has more or less knowledge on a field or with a specific tool/framework.
You can learn almost everything alone - I mean learning from the web. There are great forums, groups, discord chats, ask LLMs carefully and check on the answers. It may sound reassuring that someone watches your back and won't allow mistakes or would help clean up a mess, but you should not keep relying on this anyway. Learning by doing and taking responsibility will make you much more self assured, which is actually most of what makes someone senior.
You can have bad mentors/seniors, but looking for people to learn from and bounce ideas off is always a good idea.
Having a good mentor or two is pretty essential because most of knowledge isn't written down and retrievable by LLMs or about some framework or tool. It is the experience of people who have been there before, done it, got burned and learned to not do the same mistake again.
It might be a personality thing though. I’m a stubborn idiot who questions everything he’s taught. I don’t take advice. I listen to people stories. The why has always more important than the how.
I’ve been fortunate to have many teachers and mentors, but I didn’t seek any of them out and none of them guided me. They were people with the right perspective and I had asked them the right questions.
But the two best mentors in my life are my two best friends. All three of us are different and have different things to teach each other.
But if you are the only technical person around, who is going to show you what a good or bad practice *in your specific field* is? That you won't find on Stack Overflow or by asking ChatGPT.
Being able to talk to an experienced mentor who knows the field you are working in is invaluable. Unlike learning some framework or design patters or what not, this information you won't find anywhere else.
I've found that one of the harder aspects of being unguided is figuring out the unknown unknowns.
You might stumble into a solution of sorts that mirrors a best practice but not know there's a "name" for that solution -- until you see it spelled out after googling around. That discovery can lead you down a rabbit hole where you gain fuller context.
Sure, having more experienced people around can help expedite that process in some cases, but then again you're limited by what that person has experienced. There's always some level you reach where you need to be curious enough in your explorations to seek out the next layer of knowledge in a self-directed manner, and the tools today are immensely better at supporting that process than 10-15 years ago.
fuzzing
unittest
scm
code coverage
any more im missing?
Fuzzing is only really useful for a very narrow range of analysis scenarios. If people understand threading properly: code should be able to take getting hammered, exiting gracefully, and cleanly get re-instantiated.
Also, banning hosts/accounts with an error-rate quota system is more common these days. =3
this trend in programming culture reduces our ability to do automated error detection!
you make a good point, and a good case for crash early and crash often -- with choice of erlang style recovery, or fuzzing style hard nosed correctness enforcement
But I think there's a difference between someone who 'teaches' you and someone who is at a high enough level to discuss ideas or issues with you.
For example, in my first job I 'learned' the most not from the senior engineers around me but from another new grad who was just as curious as me. We would discuss issues we're having, brainstorm solutions etc. It was not so much that I learned from him, but that I had another brain of similar competence that I could fling ideas at.
Not having such a person makes your job more difficult (as you might find yourself in a rabbit hole and desperately need someone else's perspective) and much less enjoyable.
Imagine not having a competent dev in your team to review your code.
is not a fallacy because it doesn't work, it's a fallacy because such teams rarely exist. I have worked on one or maybe two such teams in my career, but the truth is to get on a team like this you already have to be exceptional.
If you require a team of elite experts that are not only technically excellent but also love teaching untrained juniors in their ways, then you it may take you longer than the typical span of such a career to find these people. Then, even if you do find such a team, they're extremely unlikely to take you under there tutelage unless you yourself show an impressive potential to learn. And people that are truly exceptional don't wait for the right teacher to start learning.
I started tutoring my roommate in college, and random people in the computer lab while he was finishing his homework. So I’ve been doing it longer than I’ve been professionally programming which is a long time.
https://onlineteaching.umich.edu/articles/the-myth-of-learni...
There's a good back about how to learn effectively written by cognitive scientists:
Make It Stick: The Science of Successful Learning by Henry L. Roediger III, Mark A. McDaniel, and Peter C. Brown
Definitely worth a read, even reading a summary on the book would be beneficial if you want to become a better learning. The techniques in the book work IME.
Throughout the book you keep hearing stories about how application and spacing are critical for reinforcing concepts.
I don't have it in front of me but I remember they presented one study that students who took a practice exam did better than those that just read the material and took the exam.
In the book they mention often that rereading material isn't useful or an effective way to learn.
Whereas most advice you read on social media is to pay for a course.
It seems the simplest advice would be to just read the documentation then make things, when you struggle refer back to the documentation.
Spaced "repetition is the mother of learning" is what my parents always told me. Minus the "spaced" part.
And yes, that's how we learned for most (good) university courses. We had "labs" every week. It was non-mandatory and while some profs would give you credits towards the final grade if you took part in them, the idea really was to get you to practice things over and over with some time in between. By the time you were learning for the final exam, it was already at least the second or third repetition of applying the knowledge.
Reading docs (or the book / your lecture notes) when trying to apply the knowledge is OK. Just reading it multiple times but not actually trying to use it really doesn't do anything in my experience. I passed my initial "learn a programming language" course at university without ever going to a lecture past the first two for example. I was using said language to actually build something that required what I knew was gonna be the main part of the exam (boolean logic - duh - and concurrency - way more fun) for building some actually useful tool for a friend.
This assessment totally devalues the experience of having an more senior, more capable colleague to learn from. They are not there to watch anyones back, stop mistsakes, or clean up anyones mess. Rather the point is that the junior member is able to watch the senior person, observe how they solve their problems, and how they approach fixing their own mistakes. It's about tacit learning, not bein nannied/micromanaged.
Watching a senior is not a senior pointing you to pitfalls, or unknown unknowns.
The parent argues that although there is value in being near a senior, all the actions lay with the junior. This is a transition people need to make when they leave school and enter work.
There's a big difference between learning from someone and having someone teach you something. The latter expedites your progress and clarifies learning path, whereas the former can even waste your time with political fights pulling you into dead-ends.
Keep fighting the good fight.
I would like to stress that your employer should, and will likely be happy to pay (or give you the time) to do many of the following. Don't feel like you need to do all of these by any means, I just wanted to give you everything that came to mind that has helped me in my journey:
- Conferences both in your specific domain (in your case oil and gas), or your specialty, like a more general data science conference. Meet people there and keep in touch with them, both fellow attendees and also speakers and people who have booths there. Your employer should be cool with covering one or two of these a year. If they're reluctant or on the fence it has helped me in the past to writeup a little summary of the conference, what I hope to learn and how it apply to my work, and give them a written summary when you get back including concrete things that you learned that will help your company pragmatically.
- Meet mentors on platforms like this, foster and build those relationships. It seems like you are already doing this well. Other places to meet them are at local and virtual meetups for the software or packages you use, tools you use, industry societies, etc. Linkedin is also an underrated and underutilized tool for this IMO.
- Free or low cost remote courses. I'm just wrapping up a remote class at Stanford, it's been awesome. Many universities offer courses, or even course materials and videos you can get your hands on. Good universities, with good courses.
Do these things during the work day, because they are work. Don't think like you need to do all this stuff above and beyond. Any good company should be investing in you.
-
It sounds like you have the initiative to push your own boundaries, and that's what matters most. No matter where you are, learning starts with your willingness to learn.
I'll admit I've never been in your position precisely. I've always worked with technical people, even if not for "tech companies." But, before I had even graduated I was the only person in my entire company doing mobile dev, so I know what it feels like to be very green, and very far out on your own island. It was a great springboard. I never did mobile dev full-time again, but I experimented and explored stuff I found interesting, and the skills I learned have been invaluable to my career.
That’s it.
Read books to understand fundamentals. Create projects to practice and expand on the fundamentals. Watch videos to learn how others solved problems. Document your learning. Repeat.
You'll get to work with others who are more senior than you and see how they do things. It's definitely helped me a lot.
I find that it is refreshing to hear the perspectives of working and aspiring programmers, and I stand to learn a lot about the field outside my little bubble within it: what various startups are doing, the problems they're trying to solve, etc.
In short, if you cannot find humans to talk with and bounce ideas off of at work, do so outside of work (but be careful about your company's trade secrets, etc.). Do not rely on ChatGPT. ChatGPT is to discussion what ultraprocessed food is to food.
Find other people working on similar/related topics, connect with them, offline or online. Do not fear being THE Tech Guy in the company.. just do not be over-confident / arrogant. Doubt quick-wins, check, prove things. 30 years ago such one-programmer-per-company was a normal thing.
My reading recommendations are below, but that's just what i have found good.. not exhaustive list. i found some of those only after 11 years doing stuff by myself, and saw i have been inventing hot-water in many "wrong"-but-somehow-working ways, so what.. Better than only applying miracle recipes you don't understand.
https://www.svilendobrev.com/rabota/
https://www.svilendobrev.com/1/
And, most important, have fun!
You can use a lot of modern Internet resources.
I got a Learning O'Reilly Account. There you can find courses, videos and live events where you can ask questions. It's helpful.
Read a lot of good books/e-books, ask in forums and maybe in your town is a user group for that.
Just write the software and constantly look to improve. Your code from 2 years ago is in an ideal situation, much worse than your code now.
If you aren't experienced already it is a very hard position to be in, with a huge responsibility and potential to screw up due to inexperience - and be promptly thrown under the bus when the proverbial shit hits the fan. Code reviews by ChatGPT can't compensate for lack of more experienced colleagues.
The best way to grow as a fresh grad is to join a medium sized, established business. There you are going to be a part of a team where you will get to learn the ropes of your field (that's something you won't find in ChatGPT or university) and at the same time the pressure won't be so hard. And while you aren't going to get the perks of working at huge companies like Google or Facebook, you likely won't have to deal with assorted corporate BS that comes with it either.
Only once you get a few years under your belt think about startups, small companies, etc.
> The best way to grow as a fresh grad is to join a medium sized, established business. There you are going to be a part of a team where you will get to learn the ropes of your field (that's something you won't find in ChatGPT or university) and at the same time the pressure won't be so hard.
My experience differs: already at medium sized, established companies there is a lot of corporate bullshit. Also, at small companies, you know the colleagues better, so that the culture is often more "humane". Also, exactly because at a small company you are a "jack of all trades", you by necessity learn a lot more; but be aware that on the other hand you won't specialize so deeply into the topic. So, if you want to deeply specialize in a topic, you better look at a medium or large company (but be aware that there might exist few or no jobs there that allow you to specialize in the topic that you are into).
Why?
I once encountered a tech lead who was at the same job, since graduating, for about a decade. All of the novice mistakes showed. Because there was no one there to correct basic security mistakes, the lead grew very accustomed to getting their way and getting constant flattery in the process.
When the time is right, seek a position where you're generally the most junior employee, and seek to learn from everyone around you. It's important that you can learn from others before you get into a leadership role.
I've spent my commute time to learn about these topics from youtube. For example watching EEVBlog or Uncle Bob or whatever. Then stop sometimes and just think about it regarding my past experience. Replay the stuff that I lived through while thinking about what I heard. Then find my best colleagues and sincerely ask them what they think about what I was concluding just then.
Also I like Twitter/X after just muting/notinterested-buttoning the people who just post idiot stuff. Then it is quite a good source of ideas about the present mainstream - that's also very important.
But the most important: never forget while swimming in ideas and memes: take some minutes sometime to asses what actually worked - and what is that somehow never works for ... reasons.
Great question on HN!
It wasn't until I started going to conferences, Lisp/Python/Perl etc, that I actually met people with the same level of brain damage or worse than me.
Both paths lead to the same place, having access to mentors gets you there faster I guess.
Failing that, I’m not sure what your relationship with your manager is like, but in most tech companies, part of your manager’s job is to help you grow. Maybe they could help you reach out to the few other tech people at the company to organize some kind of collaboration to help you all grow and improve.
Most of which can be distilled into: Don't follow fads, the latest shiny tech; they will only break your heart and maybe ruin your career. You want to deliver a quality product that just works, never breaks down, and if it does it is easily diagnosed.
What that means is (a) use software from vendors you can trust and have actual live support (this means they are usually expensive) (b) KISS. In a small shop you will be responsible for everything. You want as few moving parts as possible. You especially want to avoid all custom integrations; they suck. If software B doesn't offer an out of the box connector to software A which your company already is using. They are immediately ruled out. (c) never trust a salesman/saleswoman. Before you sign any contract, you want to try it out as proof of concept. Just because they say they have an out of the box connector, for example, does it actually sync the fields you want? Verify it works before signing any contract. If the salesman doesn't like it, tell them good bye.
When setting up my current operation, I needed a job scheduling framework. I looked at Airflow, but by the time I had to choose a DB backend and create usernames and set up SMTP server, I gave up. In half a day I knocked together a basic single machine scheduler, which does 5% of what Airflow does. But it does that very well, took me frankly less time than setting up this behemoth, and works really well for my team.
Is it better than Airflow, or Prefect, or whatnot? Surely not; but it saved me time and taught me a lot about job scheduling.
My tip is to try and best practice everything. Locate the correct method online, especially if theres applicable vendor best practice. Every time the director asked me to do something new, I would go away and find out what I should be doing, and then made that the basis of our negotiations. I didn't win every one of these fights, but I managed to spin the job so far that I did IP telephony, Security Cameras, built the company's ad forest and hundreds of other small relevant to my career wins. I ended up with 2 direct reports. And in the process I took a very immature business and injected a large amount of maturity that they kept with them for more than a decade. And I did them correctly, or as correctly as I could without support. By the time I looked to move on I was immediately made a senior in my next role.
1. Turn to online communities to help you learn. Find the appropriate Reddits and Discords, hang out on Stackoverflow, join Bluesky/Mastodon and follow a bunch of people in the same space as you.
2. Start teaching others. The best way to learn is to teach. Answer questions on Stackoverflow that you know the answer to, help people out in Discord.
3. Start a blog! Even if nobody ever reads it, writing about what you're learning is an incredibly accelerant for understanding things more deeply and retaining more knowledge.
4. Conferences. Ask your boss if you can get a budget to attend conferences relevant to your work. See if they'll shell out extra for workshops.
5. Side projects. If you want to learn something new and you don't get the opportunity at work, see if there's a small side project you can spin up to start experimenting with it.
6. Find people in similar roles to you at other local companies and see if you can get into a pattern of meeting up every month or so to talk about what you're learning and share notes. Obviously don't go sharing trade secrets with the competition, but in my experience there is always a huge amount you can learn from talking to peers in other companies, in an environment where people feel safe to let their guard down a little and talk openly about problems they are facing.
Branching out like this is critically important even if your company has senior folks who can give you helpful feedback, since your own company will have its own biases.
You don't need to FOMO that, really. It works like that in theory, in practice... In about 2 years of work in a midsize company, I learn about one or two little details from the tech leads in the company. Two third of them were spending more time asking why one would think there are worth asking them a question than replying the question itself. The level of rockstar insufferableness was off the charts.
Here's my guidance from 30 years in the industry. Stop doing this.
You probably do want to learn some of the basics and force yourself to adhere to them until you are experienced enough to know when to break the rules. Definitely use version control, look up typical styling/formatting/etc conventions for the languages and tools you're using, probably write unit or at least integration tests, prefer to use libraries when possible for non-trivial problems that aren't core to what you're working on. If you want exposure to this I recommend joining a quality open source project's developer discord/slack/whatever just to see how they do things. You could also maybe convince your boss to pay for consultants or for you to attend to conferences or whatever if you want individual feedback (if I were your boss, I'd see this as a good compromise - you get some level of senior supervision for $2k/yr instead of $200k/yr).
My number one advice is to NOT rely on LLMs for things that they're not smart enough for, and to always make sure you actually understand what it's doing any time it creates code for you. This is the number one issue I see inexperienced engineers making these days. ChatGPT is great for converting simple to moderately complex requirements into simple to moderately complex code under a certain size, but it's NOT GOOD at giving reliable high level advice that is "non-obvious" and unlikely to be in its training data. So please do not rely on it too much to explain complicated things to you (unless that thing is very well documented on the internet, which you would have to verify by searching for it anyway) and don't ask to eg tell you if your unit test is good. It can certainly save you a lot of time and is often right, but it's also often wrong in ways that are very difficult to spot unless you know a lot about the subject, and eventually (if not already) you'll need to advance to the point where it can't help you with your problems.
It's probably not polite to ask, but I would try and form and answer the question of why that's happening - while it can be a learning experience you dont want to be the smartest person in the room (ever) when you are just starting your career.
So in short, see if you can contribute to a well-run open source project with a good community.
Something that turned out to be extremely valuable was having the chance to make lots of technical decisions (many big, many teeny tiny) and then stick around long enough to evaluate them afterwards.
A lot of things worked out great, and a lot of things didn’t! And now I have a decent amount of insight and opinions on those sorts of things. And those insights getting me a lot of respect at my new job! Which is only my second job, after being in the hot seat at my first, which was a startup.
So my advice is try to at least stay around to evaluate your own decision making. You will end up challenging a lot of your own beliefs. I think it changed me more than anything else at that job.
0. Writing a lot of bad code.
1. Reading a lot of good code.
2. Get a textbook for a subject which you have taught yourself. Read the table of contents. Assess which topics you know cold, and which ones you have blind spots in. You can assess your mastery of a topic this way.
3. I learned the most by making bad choices and then having to live with the consequences. I learned the second most by having to justify the corners I had cut to other adults. Neither one is fun, but both teach you to be better.
You have been given an opportunity to build and lead team, hire the person you would want as a technical mentor. Focus on the big picture results, what will you be able to put on your resume? Built and Led a team that developed a technique/software/product that yielded $x result for company, or “worked on x product in y language”
It’s a different class
If someone who has a successful company wants to interact directly with you, learn everything you can from them, that is an opportunity greater than any tech bullshit
> fresh graduate data engineer
to the company, they see this as "cheap labor"
> hired 6 months ago as a freelance
basically, they put you through the ringer without having to pay out benefits
> essentially functioning as a tech lead
a fresh out of college graduate is far from being qualified as "tech lead", you are out of depth here
> with full responsibility and ownership of designing, implementing, and hiring for the projects I'm assigned
basically you are a 1-bus factor team. hope you have negotiated pay accordingly.
> Our company is not a tech company, so I only have a couple of tech-oriented colleagues
you are a second class citizen at this company, your budget will be constrained, and be on the first to cut when it gets tough. In the O&G industry, the boom and bust cycles are quite frequent.
> Now I directly report to the director of the company
now there's internal pressure to perform at a much higher level than your experience.
If you want real advice, get the hell out of this company while you can. O&G industry itself is notorious for back stabbing/conniving.
Just to clear up some misunderstandings (caused by my own lack of information supply TBH):
- I'm hired full-time with benefits now after working freelance for 6 months - I get paid 4.5x more than my peers in this company - I got these many responsibilities after proving my competence and delivering on reliable solutions in my freelance tenure. - Yep. Once again, I'm a little overpaid. - Also, currently I am being treated quite well, and definitely not like a second class citizen, since the company is making a shift towards a tech-first and data driven approach and I'm at the centre of it.
You have definitely made a few great points here though. I'm out of my depth here when it comes to management, very high level design, etc - but given the scale of operations so far, I am quite sure I can deliver outcomes!
Also, thanks a ton on warnings about the nature of this industry, I'll definitely keep it in mind!
The Oil & Gas industry is not sexy and never will be, but it pays well above market and people with niche skills will do very well there (as you have no doubt now seen).
Technological progress in the space tends to move pretty slowly (especially on the Operational Technology side of the business) owing to many factors, but mainly risk avoidance. The flip-side of this is that once one company adopts a technology, the others tend to follow suite, so what you're doing may be very portable should the need arise.
Another benefit of O&G is that due the the specialised nature of a lot of the roles, you'll keep running into the same people at different organisations throughout your career, so keep your keep expanding your network along the way.
Don't let your perception of slow growth worry you too much - on the technology side, progression will come naturally as you take on new and more challenging projects. As a tech in a very different field, but a similarly isolated environment, I too have had these thoughts throughout my career, but remember there is a big wide world outside of silicon valley, and plenty of successful careers to be had.
It sounds like you have a boss you like who you could learn a lot from - this is huge. I'd focus on getting him to teach you to become a domain expert in the Oil & Gas space. This combined with your (relatively) niche technology skills will make you a unicorn in the future and combined with your network, pave the way should you want to become independent in years to come.
Good luck!
Best comment
your comps are with “working in San Francisco”, not with your peers
Then for your skill, understand that usually data is the most important thing in a system, especially for internal company. Learn about database design and normalization, as a data engineer I'm sure you know how critical it is. As for application level, ensure that ACID is achieved on critical-level operation, and learn maintainability.
Additionally backup / disaster recovery and/or replication also can be learned if you interested on devops route more. Having a backup (and can restore it) is a must, either you or hire someone else to do it, or hosting at a cloud.
There is truth here -- yes, OP is likely out of his depth a bit. Yes, he's likely to be seen as a "cost center" rather than an asset. Yes there's internal pressure to deliver at a level higher than experience.
But none of those are reasons to flee.
OP: I think you would be wiser to enjoy this situation, learn as much as possible, deliver as best you can, BUT be fully aware that you could be nixed at any moment and do NOT take it personally if/when that happens.
Another thing to do, on the other end of the spectrum, is not get "stuck" at this company. You might find yourself extremely successful there! But there is a natural limit to how much you'll be able to learn/grow as a solo operator at a non-tech company.
Use the time you have to learn as best you can and pivot to a role at a proper eng-focused org where you can get the mentorship you want (assuming you want that!). Find another gig before you have time to learn bad habits or get too stuck in your ways.
2 years at this company will likely be great for you and be an asset to your overall career. 10 years will likely not.
Do you want to write code, and get better at writing code? Then probably find a new job with better mentorship.
Do you want to grow quickly in this company and are okay with being a manager? Then stay on your current path, hire some solid software developers to your team, and embrace the responsibility you’ve been trusted with.
- Conferences, not my thing, but if you are new, you may learn a trick or two there. No just go there, try talking to the people. Approach a few senior looking guys and ask them for advice.
- Confs can be quite expensive, a cheaper alternative is local user groups. You can try to find the closest ones via Meetup.
- ChatGPT (yup, hallucinates but still has a reasonable answer for 90% of the questions).
As for your situation:
- You are responsible for hiring. Try to hire people with more experience than you.
- "to the highest standards of the industry" isn't that perfectionism? Most production code has much much lower quality than what we see in open source.
- I think for the time being (the job market is really bad right now), just concentrate on delivering stuff. Learn if you have some time left, but delivering solutions should be your prio #1 in my opinion.
2. Do that by reading, here and other useful places, not necessarily about your own area.
3. Start to understand your industry, its details and quirks, watch your directory, ask them questions, learn the business in detail. Even if you move to delivering IT in another industry, the knowledge will hold you in good stead.
4. Understand how to model business workflows without worrying about technology. Learn how to communicate that with your users and customers (two different groups sometimes).
5. Nouns are more important than Verbs. Concentrate on the "things" and how they react to events and generate their own.
6. Technology comes and goes (MVC, DAO, microservices, monoliths, k8s, etc etc), processes as well (6 sigma, CASE, Agile, UML, etc etc). Learn their important details and differences and don't think any of them are silver bullets.
1 & 2 are the most important.
- I'm quite well paid here (4x more than my peers)
- I have been hired full-time, and get all the company benefits as well.
- The work life is great, and I don't work more than 8-9 hours a day.
- The manager/director is great and super supportive.
- I'm not being exploited by any means :)
Kindly upvote so the disclaimer stays on top of the post!
1) Read code and spend an hour a week keeping up with developments in your immediate field. Understand the most used frameworks, learn how they work, what each one's pros and cons are, the lingo, the discussions.
2) Understand what your manager cares about and what he wants from you. Ask him what would be a success and what would be a failure.
3) Make sure you understand and implement best practices. Do you have a golden set? Tests? Monitoring - Do you know how much data runs through your pipeline, what your error rate is? What wakes you up when the pipeline fails in the middle of the night? Who needs to be notified, what are the downstream effects?
4) After you have a couple of projects under your belt - that may already be the case if you've been there for a while - consider hiring an experienced contractor for a week to review your code and architecture. You said you're responsible for hiring, I'm guessing you have a budget for subcontracting. Like with all contracts, make sure you have well-defined scope for what you're asking for.
When working in an industrial setting priorities need to shift away from cost optimized garbage people obsess over like chatGPT, and understand when you mess up people may get seriously injured or dead.
Understand shell-offshore standards compliance rules, and explosion rated equipment certification process. Also, integrate the safety procedures in place so when accidents do happen people know the steps to enter safe-mode, and who to phone etc.
There are a lot of unspoken rules, and corruption is commonplace in many places.
To be honest, understand most petroleum related work will be a knock off of Kongsberg and or Rockwell Automation products. There are also dangerous crazies insisting they know something by skimming stolen CVs etc.
Consider the clowns that accidentally cut someones arm off naively touching a hydraulic control-system under maintenance (true story.) Inherent safety is the #1 skill you need to work on, catch the worst case failure modes with sanity checks, and user interaction logging for accountability.
YMMV, and no I don't work for free... Have a great day, =3
People are fragile, maintenance/inspections need done, and team communication is more important than any lock with a post-it note. People mess up, and other people bear the consequences. The "figure it out as we go crowd" is not welcome... =3
You're supposed to remove your lock when you finish up, but it sounds like sometimes angry phone calls have to be made.
Study the old engineers that still have all their body parts, and never feel pressured to do something silly.
Best of luck, =3
- Learn the domain you are in deeply. Talk to the experts (not senior engineers, I'm talking about people doing the work) in your company about how the business and industry works.
- The interface is the product. Design beautiful interfaces that match people's mental models. A beautiful architecture needs to support these interfaces. If you have poor engineering, it will show through the product via the interface.
- All good engineers are good architects and product people. They understand the needs of their users. They understand their psychology.
- There might be software patterns for your industry, find them and learn them.
TLDR; The coding part is easy, don't seek the senior engineer to get ahead, look for the industry and business experts. Learn how to build products that are impactful for the people using your software and the business. That's how you become a great engineer.
Self-directed can absolutely help. (Though, a recommendation: do a lot more reading of human-written articles and a lot less chatting with AIs.)
But you also need to find some other people to talk to, both peers and people more senior than you along one axis or another. You'll learn from them and they'll learn from you. Open Source projects, communities of practice, professional organizations, friends in other companies, online communities, etc.
As the most senior technical person in the company, you can treat this as an opportunity to create paths for technical and process development for your technical (and non-technical) colleagues. If you find opportunities, make those opportunities available to your colleagues as well, or pass on what you learn in one form or another.
There's no way not to learn from this, and there's things you can learn here that you wouldn't learn from years of being in a regular team. So just as you're missing out on valuable experiences here, you'll also be missing out on valuable experience if you change jobs.
The interesting part of a career is also all the non-technical people we meet, and if you're learning from them, that's also good. Maybe this road will take you somewhere more interesting than spending a lot of time on code reviews.
(PS: If this is actually in oil, make sure you're getting paid well.)
“Software standards” come and go, changing relatively quickly, so don’t sweat it too much. You will learn from mistakes, and that’s ok (for me, dunno for your employer :P)
I wouldn't worry about this too much. As long as what you're doing seems to vaguely match whatever information about best practices you can find online, you're probably good. As long as you're not inventing your own security tech, using floating point values for financial data or reusing feature flags in a high frequency trading scenario, any bad decisions made along the way probably won't be catastrophic and are more likely to be learning experiences.
More eyes on code is a good thing, but can go too far where a person is cargo culting their beliefs and subjective experience as objective things, leading you to learn the wrong stuff, or at least to a degree where you have an irrational attachment to some practices (e.g. some people always lean towards EAV in DB design, even though that practice has a time and place, same for microservices vs monoliths vs modular monoliths etc.).
> ...and might be missing out on a lot of learning.
This is probably what you should focus on. See if you can attend industry events, watch some talks done by people who've been in the industry for a long time, there's actually lots of nice stuff on YouTube that's not just advertisement for the latest technology or practice. Here's some people whose content I enjoy:
Dylan Beattie
Venkat Subramaniam
Kevlin Henney
A Philosophy of Software Design
Algorithms
Building Secure & Reliable Systems
Clean Architecture A Craftsman Guide to Software Structure and Design
Clean Code
Code Complete
Design Patterns Elements of Reusable Object-Oriented Software
Next-Level Database Techniques For Developers
Patterns of Enterprise Application Architecture
Peopleware
SQL Antipatterns
The Clean Coder
The Mythical Man Month
The Pragmatic Programmer
Working Effectively With Legacy Code
Also, props to whoever mentioned learning by doing in a low stakes environment (side projects) and online communities!
I think I read about this story
Sticking with tech sounds like a safe option, but if this opportunity was given to you, maybe you could think about rising in the rankings within the oil industry.
I’m not saying this is the only choice, but this is a choice that you should consider as well.
Also, learning from more senior colleagues is really a hit or miss. I’ve been in environments where I couldn’t learn anything from senior colleagues and where I learned a ton working with relatively junior people.
Despite the technical knowledge is important, the domain where this is applied is even more important. It is always good idea to keep oneself in and field, instead of going from fintech jobs, to retail, to B2B SaaS...
One of my colleages has always been working in the energy market field, and he is way more knowledgeable about everything involving the business than any other technical peer. Also, he has a name in the industry, and is known in most of the energy vendors of the country. It's reputation alone has made more than one contract for my current company, so it really pays off. Of course, he is also paid well above any of us.
If you’re the smartest person in the room, yea, that might be a red flag.
But, here’s some different advice though: get the company to hire someone more experienced. Be willing to give up some oversight responsibilities to that person. (Or hire someone who is experienced and explicitly doesn’t want to play the politics game.)
I felt like I was in a similar situation fresh out of school. It wasn’t until my company hired an incredibly talented person that I felt I was finally learning from someone (beyond reading books… back in that day, they were still pretty relevant and up to date).
That being said, the one thing that you can't really replicate is getting honest feedback about what you're doing at work.
For example, you can learn all the design patterns, write the best code etc, but are you even working on the right problem? What are the assumptions that you've made that are incorrect? Is there a much simpler approach that'll get you there in one tenth of the time?
These are thing things that experience may provide which no course or single book is going to give you.
You basically need senior people to talk to.
The most straightforward way to do this is to get a job which has this structure set up for you already. Otherwise make friends, go to meetups, do some networking and find some experienced people you can get some feedback from.
Not impossible but much harder than working closely with humble and experienced engineers. That's why I recommend that a junior engineer finds a great consulting company as soon as they can and spend a few years there. This was a kinda breakthrough period in my career for both technical and soft skills. The nature of consulting also means that you might see a greater variety of environments, technologies, and problem/solution combinations than the equivalent xp in a non-consulting role. But make no mistake, finding a "good" consulting company is key. You need to have humble and experienced engineers who want to help you and grow you. If it's a "bums in seats" kinda "body shop" or some kind of celebrity house for "hero" or "10x devs" then maybe this won't be as useful.
I work in a team with some very senior colleagues (and even famous in computer circles for a couple of them) in a reputable company. I actually don't learn much from them. They review my code, we discuss some design docs, it's certainly interesting to see how they manage their time and how they deal with various problems, but I don't learn much from them in the technical sense. Also to my surprise, we also don't produce software to the highest standards...
If you want to improve technically, you can read books on things you'd like to improve, or find technical projects outside of work.
- How will the company / your boss decide that you are doing a good job? - Focus on that first. Include the parts they don't directly value but where you think you need them in order to reach that goals.
- How do YOU know you are doing a good job? - to develop that taste you need peers to look up to. Find them in meetups, in un-conferences, conferences or on the internet.
- Build your skills in three main areas:
- "Behavioural" skills (a.k.a "soft" skills) - written communication, verbal communication, be good at giving and - domain skills (learn about the company & the field)
- tech skills - focus more on concepts & principles and less on the "framework of the day".1) Start contributing heavily to a popular open source project you're familiar with and use. Try to make your PRs as high quality as possible. You'll get free code reviews from some of the best engineers in the world doing this. That'll be a million times better than ChatGPT code reviews and you'll be able to learn a ton from it while also getting your code into production at thousands to millions of companies in the world that depend on the open source project/s you choose to contribute to.
2) Fill holes in your knowledge base. If you feel weak in a particular area like networking or DSA then study and practice with it until you no longer feel weak in that area. If you were on my team I would try to assign tasks to you to help you naturally build out knowledge and confidence in your weak areas but without someone to do that for you, you'll need to try to figure out your own weaknesses.
3) Always try to do your best when working professionally. This is all any of us can ever do and if you actively practice it then it'll become a habit that will guide you toward success in any environment.
Not sure about the other points.
I wouldn't say you'd only learn specifics of the frameworks. Maybe if you only submitted 2 or 3 PRs, but that's not what parent is suggesting: 'start contributing heavily'.
As the other poster pointed out, heavily contribute implies the project still could use some major new features and there is plenty to do. Even porting something KDE to QT6 would be a major effort and teach a lot even if you don't write any new code. There are projects that are complete as well and so there isn't much to do.
But, sure, don't send PR's for things that don't matter just to send PR's, that's just annoying.
But if you send a 3-line PR to fix a bug, that required 15 hours of study of the code to diagnose and figure out the right 3-lines, that is HUGELY educational.
Working on immature or even poorly written software can be super educational too, nothing wrong with that too.
But if you only ever look at hacky not-yet-mature not-yet-proven software, how would you ever learn to write high-quality software, or even know the difference? You have to study known good code. The way programmers study something with a natural goal instead of just artificially, is in the course of trying to alter it "correctly", the study you must do to figure that out.
(I know some programmers who have literally never worked on anything but over-engineered architecture-switched-every-month-for-years barely-holding-together technical debt bankrupt software... and I think sometimes some of them literally don't know anything else is possible! Looking at mature respected code is crucial, if you don't even know what it looks like how could you possibly try to get there?)
Getting your own work critiqued is unmatched, but reading what others did is the next best thing.
It wasn't fun, but I'm better for it.
1. Learn algorithms, buy some book and start reading.
2. Learn theory of neural networks, use the Chollet book to develop some knowledge and intuition.
3. Write many small "pet" programs implementing basic stuff: implement a small database, a small programming language interpreter, a small editor, a small neural network, ... Each time try to apply good design.
4. Embrace simplicity in everything you write, don't make things more complex than needed.
5. Read good quality code, especially read open source code which is at a degree of complexity that makes reading valuable. Dont' read Kubernets... or PostgreSQL perhaps. Learn more self-contained code bases.
6. Participate to some OSS poject.
7. Start some side-project and put it on GitHub, were you are the main designer. Develop something that you need. A library, or a small utility, something that you really enojoy doing. Do it at a quality level that makes you happy. Never regret pushing code on GitHub if you feel that for the level you are right now it is appropriate.
8. Don't give a fuck about what other programmers think of your work, if you did it at the max level of what you can do. Anyway most programs on the Internet suck, including the ones of people feeling very competent.
My thought process was I get to learn stuff, so even if I didn't make any money off it (I mostly didn't), I learned how to ship. This included (yuck) documentation, QA, etc. Doing things like that you really learn not only how to code and how to structure your programs, but about the 80/20 rule which to me is it takes 20% to get POC and 80% to get it ready to ship.
The senior I finally got to work with was mainly to validate that I was on the right path, because I could write just as well as he could. I met this guy probably 4 years into my career and we were pretty equal at that time. He had about 5+ more years experience than me, and he was VERY sharp. I probably haven't worked with someone better since and this was 20+ years ago.
One more thing, I was the "computer guy," at my first company out of college. I did the workstation building and support, the networking, the server setup and user administration (Windows NT days), and wrote applications in MS Access for the company to use. I had to interview the users and managers on what the apps should do and whatnot. I had to learn how to scale MS Access for 250+ user base (ugh). I had to do all the reports and everything computer related. That company was a toxic shithole but it was actually a blessing. I never would have learned as much had I got a big corporate job. You tend to get pigeonholed in those places doing the same thing over and over. And yes, I was terrified of not being good enough the whole time; that helped too.
Good luck!
Standards can slow you down, which is fine in a big tech company, but probably not good for building value for your company.
It's better to think more about what may break in your code base that could be emberassing for your company and build integration / othere tests for those important cases.
Otherwise generally simple code is good code, keep building and have fun in life. Lot of those ,,standards'' just overcomplicate things and may hurt your company.
Because you might feel unsure of yourself, you will be tempted to hire folks who are less junior than you - resist that. Hire senior people (budget allowing) and start your interviews with them by asking if they are comfortable reporting and working with someone with less experience than them. As long as you are humble and want to grow, you will.
I think the recommendation to hire seniors and learn from them is great, but I would not muddy up waters by asking the above.
Most senior engineers have reported or saw colleagues report to non-technical leadership. Senior is practically synonymous with being able to make technical decisions without technical guidance (only business guidance). Asking that as a minor question in the middle of the interview is OK. Making that sound too important may just raise worries that the interviewing company had some people issues and might be a mess to step into. My 2c.
Ideally you should also run those linters in CI/CD. When a new member joins the team, they will get CI/CD linting for free, which will save you a ton time for each new team member
My first thought is... why? Have you tried reaching out to them to discuss what you're building? Maybe a shared chat channel (slack, teams, whatever) where you can discuss what you're doing; or even rubber duck and people can read as they want and offer their insights. Just because you're not on the same projects doesn't mean you can't talk about your work with each other.
I don't know what kind of languages/environments you use, but I recommend investigating and trying out some popular technologies and architectures that are allowed in the scope of your work. Try to understand the pros and cons of each approach.
When you want to know the "right way" to do something, try reading the code of the most experienced programmers and successful projects written in the same languages/environments as yours. Very little in software is "new" so chances are someone has tried it before. Maybe LLMs are good for this, but I usually just read the code.
Focus on iterative improvement not destinations, because you'll never be "done" you just move onto the next project and the next technology. You'll get better as long as you're trying to learn new things.
Keep a tech journal.
In it, describe what you're working on, what problems you're trying to solve, and questions you have.
Make conjectures based on what you know and what you guess, and write those in the journal too. Come back later and flesh out or correct those conjectures as you learn more.
As your writing reveals holes in your knowledge, add those topics to a list of things to research.
This kind of journal is much easier to keep in a wiki or note-taking app like Obsidian, since you can then build a personal knowledge base around it. However, you can also do it the old-fashioned way, with pen and paper. Do whatever comes easiest, since that's what will stick.