Technology, on the Stack

Following Ted's advice I'm taking a short break from writing code... to write an article about writing code. Story of my life.

My day job involves coding. It's a great workplace but there's only so much satisfaction I can get from a job where I'm told what to do. Cardsphere is my real passion and my creative outlet. Combine that with the fact that Magic's my favorite game, add like-minded partners into the mix, sprinkle in all the encouragement we get from the community and baby, you've got a stew going.

I've been caught in the 'sphere for more than a year now, exchanging months of my life minute by minute for the privilege of serving this community. Being a father of two, balancing two jobs and the family is hard. If not for the support of my dear wife this would have not been possible. I'm sure it's the case for Tom and Ted as well. We have to find ways to be efficient with our time and resources.

Resolve the Stack

Luckily in 2017 the available technology is very good. A small team can achieve in short time what used to take years, and on a shoe-string budget. For us deciding factors often came down to the lowest common denominator between me and Tom in terms of the technology we were comfortable using.

• PostgreSQL - you can't go wrong with this one. Features, performance, stability far outweigh the few shortcomings of this amazing database system.
• Python 3 - because when your partner wants to code in Python you'd better make sure it's Python 3.
• SQLAlchemy - the true reason to use Python for your server projects. This object-relational mapper is unmatched across programming languages in my opinion. Feature rich, flexible, predictable. What else do you want from an ORM solution?
• Alembic - because you always need database schema management to go with your database.
• Flask - when you are on Python but can't stand Django.
• jQuery - because choosing a Javascript framework in 2017 was harder than ever. Also because it is required by Bootstrap.
• Bootstrap - yay! Responsive design.

Once we had chosen our tech we had to move fast and prove the skeptics and naysayers wrong. We had to survive the "vaporware months".

Early on Tom sold me on using Heroku to host Cardsphere. My first thought was that it was a little overpriced but it turned out to be one of the best decisions we made. Being a small team we can't afford any time wasted on devops which is the ongoing management of our staging and production servers. Heroku takes our code from our repository and does the rest so we can focus on development:

• no deployment scripts
• no messing with the operating system or packages
• no security patches
• no access control
• seamless horizontal scaling
• seamless monitoring
• seamless TLS/HTTPS
• managed database and other services
• database backups and protection

The first big challenge was package matching. We knew this was the core feature, the combo card we had to drop in order to win. With relative pricing (expressed as % of index), price limits, and wants that represent a whole group of printings at the same time, this was clearly a non-trivial challenge. We prototyped a couple of solutions with different representation of the data in the database. We were impressed enough by one of them working so well on even relatively large simulated data sets that we went with it.

The Opener

Our public beta with faux currency turned out to be a great idea. It demonstrated our progress to our future users, provided valuable feedback, and allowed us to fix most issues before going live with actual money. We've relied on our community's enthusiasm to be our fuel since day one, and real people using the product sure filled up the tank.

And so we started building Cardsphere out from there. As the code base grew we added automated unit tests to the areas of code that had to be spotless (trading, book-keeping). There was not time for full coverage, so we had to be judicious. We had to be mindful of the cost/benefit.

One aspect of the system that received a lot of development attention was concurrent access to data. What if somebody commits a trade to a user at the same time as they are requesting a cash out? What if a user is deleting a want record but it is being consumed in a trade at the same time? Things like this, simple and obvious on paper, are often rather complicated in practice. Cardsphere uses a mix of optimistic and pessimistic locking strategies to achieve this goal.

The beta was a great proving ground, showing us we were on the right track with a lot of our ideas and implementations.

Tempo Game

In terms of development process it's important to strike a balance between building bare minimum of the required functionality and spending time to future-proof the product. I'm of the opinion that small doses of future-proofing pay off many times over in the long run. It's better to spend 10% extra time to make sure you don't spend 300% down the road.

That being said, we rarely make incomplete features linger past one week sprint mark. Most of the features are developed and tested same week and go into production with the next deployment. We usually do our deployments on the weekends when we can monitor the system for any unexpected issues.

Currently Cardsphere is being run by one server node and one database node. I'm looking forward to the time when we there will be a need to scale horizontally - that is increase the number of server and/or database nodes in our cluster. We have built Cardsphere with this concern in mind, but we are yet to use it in such manner in production.

So do we cut corners? Sure.

Sometimes we do less testing than we'd like to. For example we don't normally test new functionality on all browsers. With the exception of IE11 (yuck!) all modern browsers are mostly standards compliant, so we accept this as a trade off for time. And we have you guys to tell us when we've got something wrong. You know those bugs won't sit unpatched for too long.

Level Up

As the product matured it became clear that we could do better in terms of code and technology:

• API - we started with no external API to speak of. We changed that. Most new features are relying on our new REST API. Our goal is to have our user interface use our REST API; at some point in the future we may expose all or parts of this API to our users.
• A modern Javascript framework - after some research we decided to give Vue.js a spin. It's similar to React and later versions of Angular, but unlike those it's much more transparent and better suited to be mixed with a classic jQuery based multi-page application. Several pieces of Cardsphere user interface are already built with this new technology and integrate well with the rest of existing code.

Looking Past Round 1

Each time I look at the Valakut of stories in our back-log it feels like we are still in the beginning of a big journey. I hope we all can enjoy the journey as much as the end goal.

I hope this has been an interesting behind the scenes look. If anyone has any questions, you can find me on Discord most of the time.

Thanks,

Michael.