Hello, my name is Derek Gurchik and I am a software engineer in the Philadelphia area.
Some basic facts about me:
- Some of my earliest memories with computers are me installing a different Linux distro on the family computer each week. As an adult, not much has changed.
- I am a long-time reader of /r/programming but I have not yet learned to religiously defend my favorite language/editor/widget in the early hours of the morning.
- /r/homelab inspires me daily to acquire
loud and expensive space heaterscomputer equipment first, then consider uses for them second.
- I have had an off-and-on interest in chess since high school. Since the pandemic started I’m currently “on.” That doesn’t mean I have any skill though…
To reveal it, base-64 decode it using one of the many base-64 decoders online (here’s one), then re-reverse the string. Or if you have Python 2 or 3 installed you can simply paste this into a terminal:
$ python -c "from base64 import b64decode as d; print(d('bW9jLmtpaGNydWdAc2Vpcml1cW5p')[::-1])"
Mar 2020–Apr 2021: Penn Interactive (PI)
I joined PI to work on their yet-to-be-released Barstool Sportsbook. PI is the interactive gaming division of Penn National Gaming (PNG), one of the largest American casino operators. There were two goals for working at PI.
First, after working at a large company like Capital One I wanted to try experiencing a smaller company, and at PI I was around the 25th engineer hired. I had the idea that working at a smaller company would mean that I could have a bigger impact, my work would be more appreciated, and there would be less red tape and siloing. While it is true that you can get many of these things at a smaller company, I realize now that it isn’t impossible to get these at larger companies. Perhaps I’ll write more about this later. I also think I missed some of the small company experience by working remotely the entire time as well, but there isn’t anything that could be done about that.
The second reason I wanted to join PI is that I would have the opportunity to build a lot of stuff from the ground up, some of which would use technologies new to me and would strengthen my knowledge in others. This there was no shortage of: I remember many nights working until 11 pm, and since the widespread shift to WFH just started office equipment was largely out of stock, so at the time I was working from my kitchen table (in hindsight these long unergonomic hours could explain why I suddenly woke up one morning with a herniated disc later that year). I was the first person hired on the SRE team, inheriting a Jenkins instance without any pipelines written for it and asked to get working. My only prior CICD experience was using Capital One’s automatically generated pipelines so I had a lot to learn. By the end of each Friday I was learning a better way to complete the work I had just completed that week. Terraform was also mostly to me, and while I had experience with CloudFormation I learned a lot about how to write, organize, and maintain Terraform code that was open to the entire engineering team. Later I extended the Terraform and deployment pipelines for the backend API to deploy in a blue-green fashion in staging and prod. After a successful launch and after the SRE team grew to one of the largest single teams in the engineering department, I transitioned to the Data Engineering team where I worked on mostly Python stuff: data streaming with Kinesis, ECS and Lambda, and other automation around the data warehouse.
The timing of my March 2020 hire date is interesting for two reasons. First, it was two months after PNG acquired a 36% share of Barstool Sports, so we were heads down in taking a proof-of-concept sportsbook and turning it into a real product customers could use (and do it all before the August release in Pennsylvania, a feat some on Wall Street said was impossible). The second reason March is interesting is because of COVID-19. In my last week at Capital One people were only speculating about whether a temporary WFH policy would be enacted – IT was working around the clock on an overhaul of the VPN to accommodate this shift if it became necessary, but no one knew what would happen. The next Monday when I started at PI, suddenly we were in a different world: not only was I going to be onboarded remotely (and ultimately never meet most of my coworkers in person), but it was also the start of casinos being forced to shut down by their respective local and state governments to contain the spread of the virus. Once they began the closures spread like wildfire, and PNG furloughed over 90% of their employees and had to sell some of their casinos for rent credits to keep the lights on. Even though the pandemic is not yet over as of writing this, things have definitely turned around for PNG. With the launch of the sportsbook in each new US state we continued to grab market share and compete strongly with the established players. The share price of PNG’s stock rose 965% from when I was hired to when I left and was included in the S&P 500, although I did not earn any stocks as part of my compensation package so it’s just an interesting fact.
Aug 2019–Mar 2020: Capital One
After a relaxing post-graduation break I started my first full-time software engineering role at Capital One. New graduate software engineers are hired within their Technology Development Program (TDP) where you are placed on a random engineering team for your first year at the company, and after that period you interview with other teams and select your final placement. In hindsight this was a process that I was not fully aware of and if I were to do it again I would have accepted another offer for reasons that will probably become apparent soon.
I was placed on a team within Capital One Bank SRE that was primarily responsible for on-call production support and change management for many aspects of the legacy systems within the organization. The on-call was particularly demanding: I heard it was common to receive an average of about 1 page per hour, each of which requiring triage, risk assessment, communication with stakeholders, etc. My luck would have it that a couple of months before I joined approximately 50% of these pages were reassigned to another team, however it was still the case that the team’s responsibilities were almost completely non-engineering in nature. After some networking and discussions with my manager I was given the opportunity to work primarily on the development of internal tools within the organization along with the on-call responsibilities. I had a lot of fun with this and worked on two main projects. The first was a streaming platform to ingest Cloudwatch logs to Elasticsearch, which was cool because I agreed strongly with the goal to optimize for developer convenience; by adding only a single line to a project’s pipeline configuration file, your application’s logs to stdout would be ingested into Elasticsearch, no code changes required. The second project I worked on was a microservice API that would augment the organization’s standard in-house CICD pipeline to ensure every project had a sensible baseline PagerDuty configuration; when a deployment was triggered for an application, using information from our employee database it would ensure that every engineer on the team responsible for that application had a PagerDuty account, and if no on-call schedule was configured, the team lead was sent a reminder to set one up.
I enjoyed working on internal tooling a lot, but as the end of 2019 began to arrive the department began to talk about reorganization. After the New Year the final details were announced, and as part of the plan I would no longer have the ability to work on internal tools. Instead, I would be “embedded” in a “pod” of development teams that align with a certain subset of core functionality of the Bank and I would be responsible to provide consulting on all aspects of their reliability. In particular, my “pod” was Money Transfers, that is to say, I was “embedded” on every development team that worked on systems that were required for customers to make ACH transfers between internal and external accounts. My new day-to-day responsibilities were to meet with the development teams to learn about their application’s architecture, how it relates to the overall “pod”, historical context, recent or ongoing issues, future plans, CI/CD practices, testing practices, on-call practices, toil, etc. The plan for Q1 was basically to become a subject matter expert on the entire pod, and Q2 and beyond would be determined at a later date.
It wasn’t quite communicated to us what caused the reorganization but I’m assuming it was caused by the negative press the company received in the second half of the year due to the hack as well as some outages. Regardless, there were few (if any) people in the organization who could call themselves a subject matter expert on an entire “pod” at this level, so our work had obvious positives. However the personal negatives for me is that I found this work very stressful, with no end in sight; we were already flying this plane while we were building it, and leadership did not yet know what would happen to these projects 6-12 months ahead. In theory these problems wouldn’t necessarily be mine to worry about after my one-year assignment on this team was over, but there were rumors that due to the number of TDPs hired and the lack of teams with the experience and willingness to manage them, some TDPs would be asked to stay on the same team for a 2nd year. While they were just rumors, I was very worried that if this became reality, it was going to become impossible to successfully interview at another team in the company (or another company for that matter) without having written any code in the last 6-18 months. I tried speaking to the managers of the Program about my situation, but I was unsuccessful in having any change made. I was told, “What you want to do [writing code] does not align with business need.” I did not blame the Program for this as they had nearly a thousand of us and there were only a couple of them managing it. While I understood the “business need,” when I was given the opportunity to join Penn Interactive and have a guaranteed way of writing code and learning new technologies, it was a no-brainer for me. In hindsight, my concerns ended up becoming reality as later in the year the rumors were officially confirmed.
Jun 2018–Aug 2018: Capital One (internship)
During the summer of 2017 I worked at Capital One as a software engineer intern. This was in Wilmington, and the housing provided to us was very conveniently located near the office. I was placed on a small (less than 5 engineer) team in the Contact Center division responsible for developing a proof of concept call transcription model for a large volume of customer support calls (about a million audio recordings per month). I helped with the orchestration side of this model which was written in Python using EC2 and SQS and using Elasticsearch and Logstash for storage. I improved some of the Python code and set up monitoring and alerting using New Relic to measure technical issues as well as some business metrics. For example, we were particularly concerned about how long it would take for a transcription to be complete after a call ended, so I made the code changes necessary to capture these metrics and optimized the procedures to bring this lag down from hours to single-digit minutes. As for the technical issues the monitoring was intended to reveal, I was able to find some bugs in the infrastructure and orchestration code that was causing about 10% of transcriptions to be missing due to an error, and after I fixed this the error rate dropped to less than 0.01%.
Jun 2017–Aug 2017: Susquehanna International Group (internship)
During the summer of 2017 I worked at Susquehanna International Group, a fintech market maker in the Philadelphia area. Other than saying I worked on a C# development team I can’t talk much here about what I worked on, but I was very impressed by the internship program. Housing was paid for us in the UPenn dorms in University City, and we were taken to many fun events like bowling, the Philadelphia Zoo (where I was able to ride the hot air balloon which was a first for me), Treetop Adventure Course, the quarterfinals of the Gold Cup at Lincoln Financial Field in the Presidential suite, a dinner and comedy show and poker tournament in Atlantic City, and a Phillies game.
I studied Computer Science at Rutgers University (New Brunswick) until May 2019. While I was in college I was very fortunate to have the opportunity to work at Rutgers OSS, a small team of mostly student developers in the University’s Office of Information Technology. Our job was essentially to identify a need within the university and create a proof of concept using existing open source software or write it ourselves if it made sense. For example, we used SaltStack for infrastructure management for our on-campus data center and I identified and fixed some basic bugs that we were experiencing (here’s one example). We also mirrored the software package repositories of several popular distributions like Ubuntu, Arch Linux, etc. so our servers and the student, faculty, and staff community within the University would have gigabit-speed software updates when connected to the University’s network. When we realized that these mirrors were occasionally unhealthy (for example, by receiving a poor score on the Arch Linux mirror status page) I developed some monitoring that would detect when we hadn’t synced in a reasonable amount of time and email us to that effect. I worked at OSS during the school year and during the summer of 2016, until the end of my junior year when I got closer to graduation and had to focus on my final courses needed for graduation.