State of the Sphere - November 2024

It’s been a long and short year at Cardsphere since the acquisition by EDHREC parent company Space Cow Media. There has been a tremendous amount of work happening, but to date there have not been many user-facing changes with the exception of better performance all around, but that’s not exactly what most had in mind for the first year as part of a larger company. The Soon™ discord meme is as relevant as ever, but as we head into the holiday season, I thought I should give everyone an update on what's been done so far, and what is coming Soon™.

Content Team

One of the most impactful user-facing changes that have been made since the acquisition has been the onboarding of a new content team. If you've seen the front page at all over the last six months or so then hopefully you've noticed that Cardsphere now has a team delivering excellent written content that is Cardsphere-specific on a regular, weekly basis. A welcome improvement from the content schedule before which was sporadic, at best, due to the fact that we just didn't have the time to wrangle and manage a dedicated team. Now that we have this in place, I've made sure that while we do have regular columns that you can look forward to, there is also still room on the schedule and budget for the community-focused articles that can use the blog and front page to shine the light on members of our wonderful community.

Backend Work

It’s hard to overstate how much has gone on in the backend. For those not familiar with software development, when we say “backend,” we're basically talking about what our users feel but don’t see. Database work, improvements in speed, availability, redundancy, and especially the ability to scale are all included in the backend. Boring and invisible, but vital to site operation. Users who had massive Wants and Haves lists made the package matcher struggle at times, and when they tried to download their ledgers or export their data, it would sometimes fail, and one particular user's ledger was so huge that the entire site would go down when they tried to download it. This was a pretty good indicator to us that if Cardsphere exploded overnight, it might really struggle to keep up with a massive increase in activity.

So, migrating away from Heroku and onto AWS solved a lot of those issues, but many underestimate what it takes to make a change like this, especially when the goal is for this change to be totally seamless and transparent to the users. So, I’ve brought some handy graphs and data to show just how much work has been done on the back end.

This chart shows the frequency of commits to the Cardsphere backend by month. When we say “commits,” it basically means changes that have passed testing and quality assurance and have been made part of the production product that all of our customers use. Is this a lot? Here are some stats to go with this chart:

141 commits from Nov 13, 2023 - Nov 19, 2024
Lines of code:
insertions:    49154
deletions:     6160
files:              1148
commits:      141

That’s not all. The frontend work, which will open the path for all of the exciting feature updates everyone has been waiting for, has required its own branch of backend development. This makes it possible to do frontend development in tandem with the existing frontend and without any interruption of service while we figure the new frontend out. Here are the stats for the backend work to support the new, upcoming frontend.

44 commits (branch was started this year!)
586 files changed
Lines of code:
insertions: 24367
deletions: 497
modified: 14256

The commit stats are very important. Between the production backend and the new branch, there were 185 total commits, 141 of which were made to production that everyone is currently using. In all of 2023, the year we developed and launched Marketplace, the largest update in CS history, there were 53 total commits, which hopefully demonstrates the magnitude of work that has been required on the backend.

First Look At the New Cardsphere Frontend

The frontend was originally promised to be pushed out this fall, which has almost come and gone. With these delays, I think it’s only fair to finally pull back the curtain and reveal the new Cardsphere frontend, which we've been talking about for quite a while now. Take a look, old CS first, new CS second!

Okay that was a bit of a head fake, and they are actually different pictures. The new CS looks and works exactly the same, and if all goes as planned, no one will notice this either. If it’s exactly the same then what is the issue, what's the point, and why is it taking so long?

So what’s the hold up? Well, buckle up, cause we're about to get real nerdy.

Shortly after the acquisition, the decision was made that the entire frontend of Cardsphere needed to be overhauled, done in the interest of an improved development cycle in the future. Cardsphere famously had one developer, Michael, who wrote the entire frontend in a framework called jinja, which served us very well for a while. Then we needed the help of a more advanced javascript framework, but Michael didn’t have the time to rewrite the whole frontend. His solution was to work vue.js into the existing jinja templates. He custom webpacked this old version of vue.js and applied global variables in the jinja templates to make pseudo server side rendering. It was exceptionally clever and an excellent way for a single developer to begin tacking on newer features to an old system with minimal work. Unfortunately, “exceptionally clever” for one developer can translate to roadblocks for a larger team.

To continue to tack new features onto this old system would be plunging Cardsphere deeper into what is commonly called tech debt, an accumulation of increasingly dated software and quirks in a code base that slows down development longterm. Tech debt, in reality, starts as soon as you start writing software, and to combat it is always an uphill battle.  Beloved Magic website Cube Tutor famously fell victim to tech debt. However, given the opportunity of a new team coming on that is more familiar with react.js, next.js and typescript, we made the decision to take a moment to rewrite all the frontend code in one go, rather than continue to contribute to tech debt. The result will be shiny new code that will appear exactly the same to the user, but make our devs’ lives a whole lot easier as they start hitting up our feature backlog next year.

To anyone, like me, who knows the notorious meaning of “Futuresite,” the idea of taking a perfectly good website and fully rewriting it might be giving you deja vu severe enough to make your head explode. Still, the reasoning is sound, and this will be best for Cardsphere in the long term - as long as we get it right, which is why it’s taking so long.  The truth is, the frontend rewrite is done, but it's being held up by one more thing.

Now that I'm empowered fully as Product Manager, it’s my intent to make sure there are several rounds of quality assurance that take place before we make the switch to the “new” Cardsphere. This includes testing by SCM devs, a QA pass by myself, a second round with Bodey and me, and finally a third round where we'll invite members of our community to join us in trying to break the new product, and maybe even live AB and load testing if necessary. Given that we're right in the middle of the holidays, I would expect this to take place early Q1 2025.  We will absolutely not be pushing out a broken product, and, ideally, the switch to react will be all but transparent for the users. This is what's holding up most of the feature development; hopefully everyone reading this will understand!

So What’s Coming?

It's important to know that the plan for now is to put most of development effort toward Carsdphere "Classic," which is what I’ve been calling it while this has been planned. Cardsphere Classic meaning, not Marketplace. CS Classic is more feature complete, and is much closer to a "finished" product that can then evolve glacially as most mature products do, and a vast majority of trades still happen on CS Classic. It’s beloved by users, and it’s how we are best known; it's what got us here. In 2025 we will take CS Classic to the next level.

We are not abandoning Marketplace, but it’s still very much a beta product that needs a significant amount of work. So, we want to put most effort toward CS Classic and get it to a place where it’s ready for prime time, after which we can finally introduce Cardsphere to the larger Magic world through marketing and sponsorships. Additionally, we will be focusing on bringing forth a lot of quality-of-life improvements that will benefit both Marketplace and CS Classic users. So while the major Marketplace improvements might take a bit to land on Cardsphere, all users will benefit from what's coming in 2025.

So, what’s on the table? In short, everything! Right now, in between QA and my other duties, I’m working to prioritize which features can be rolled out and in which order, based on how much dev effort it will take and how impactful it ill be to our users. We don’t have solid dates, but I can tell you that there will be some major new features coming in 2024. Yes, this year!

Probably Coming in 2024

Improved Importing Haves and Wants

Backend devs have been hard at work mapping every single card, including tokens, to the appropriate Scryfall ID, and importing will finally use Scryfall ID to match cards on CS (hooray!). This should alleviate one of the largest pain points our users face while trying to import decks and collections onto the site. This is near complete and should hit this year!

Paste Decklist and Collections for Import

Another often-requested feature, our importer will now allow you to paste the contents of your clipboard into a field that will allow directly importing the contents without a CSV. This should drop this year! Several sites do this already, including our sister site Archidekt.  Here’s how this feature looks over there:

Scryfall Search

The short of it is, we will be adding Scryfall search to our current search functionality, fully supporting their syntax for finding those pesky cards that seem to not be part of any larger set.  This should also drop this year. I intend to have search be present on every page, but that aspect of this improvement might not come until 2025.

Probably Coming in 2025 and Beyond

Development for everything here is pending completion of the frontend rewrite.

User Interface/User Experience Overhaul

Remember when I said Cardsphere probably looks plain and dull to most? While that may continue to be true, there are a lot changes that can be made to improve the user experience while using the site. This will not be Futuresite. We’ve been working closely with a UI/UX expert who helped Archidekt with their overhaul to keep the same clean, easy to read interface, while still making the site appear more approachable by making things make more sense. I’ll be the first to admit I was skeptical, but after seeing the first iterations of suggested UI improvements, I am genuinely excited for this, and you should be too.

Search on Every Page

Makes sense and we should have had it all along.  The new search will be present no matter where you are on Cardsphere, no more three clicks to start searching! This requires a frontend change, and any change we make to the current frontend needs to be rewritten for the new frontend, so there is a slight chance this shows up in 2024, but I would expect it to be in early 2025.

Wishlist 2025 Items

I haven't yet discussed fully the complexity of these features with our developers, so I’m just going to rattle off things that I would very much like to have added to the site in 2025. It’s likely that many of these will come, but it's not fair to make promises until I know for sure that it can happen, so I will say that at least some of these, but maybe not all of them, will emerge in 2025. Some or many of these features may be Premium. If you’re on Discord and have seen me say “It’s coming,” then it’s likely on this list.

Gallery view - "Booster Fun" made our current interface very difficult to use. A gallery view that shows card images is necessary.
More badges/badge progress meters - Badges are one of our most popular features, and we’ve heard loud and clear that you want more goals to chase. Will be retroactive, if possible. I’d also like users to be notified when they’ve earned a badge.
More/improved Filters - Filtering by almost anything and everything, maybe a Premium feature?
Filter presets on send page - Save the different combinations of filters and easily swap between them. Premium feature.
Creeping offers - Increase your offers by a % gradually. Premium feature.
Keep top offer - Have your offer increase if someone is outbidding you, respects your price limit. Premium feature.
Dynamic sealed pricing - At least for booster boxes, follow the market and not pegged to MSRP.

There is much, much more coming that we’re not quite ready to talk about yet. If I've told you "It's happening" in Discord, but didn't mention it here, it means it's probably part of a larger update that we're not ready to talk about yet. Still, 2025 will be a banner year for Cardsphere and I’m looking forward to finally delivering some of the features that our users have been waiting on for years.

Thank You

Thanks so much to all of our dedicated users that have helped us get here. I really can’t express how grateful I am for each and every one of you. So many of you reached out to me personally, some even by snail mail, to say how much the site and community meant to them when it looked like we may be closing our doors. To see that all turn around and find the site and our community standing at the precipice of a meteoric 2025 for Cardsphere is beautiful and surreal, and I’m so thankful to have played such a role in all of it. Thank you.

xoxo

Efren