Digital Freepen

Studying Go in Korea

Sat, 26 Nov 2022 12:00:01 +0000

In September 2022 I went to South Korea to study Go for a month. For the audience of this blog, I usually need to specify: Go the board game, not Go the programming language. In the western world, the game is called using the Japanese term Go (碁) but for the remainder of this blog post I will use the term Baduk (바둑) since I was in Korea. If I was in China, I’d call it Weiqi (围棋).

Where in Korea did you go to study baduk?

Fortunately I had a starting point that didn’t require me to know how to speak Korean. I went to Blackie’s International Baduk Academy, which is a baduk school specifically for foreigners. Kim Seung-jun (Blackie) 9P and Diána Kőszegi (Dia) 1P are the teachers.

Accommodations are provided and you live with the other students/teacher. As such, baduk wasn’t just classes I had to attend at certain hours of the day, but rather it was the entire experience. We also weren’t in Seoul proper but rather, in the suburbs at the edge of Seongnam/Gwangju. That being said, the teachers had a mini-van and we drove to Seoul quite often. It was quite impressive, it could fit up to 10 people and we even got to zoom past all the morning traffic by taking the bus lane (minimum 6 passengers).

Why Korea?

I believe there used to be places in Japan or China that foreigners would go to study pre-covid, but neither countries were open for travel (Japan opened at the tail end of my trip, and China well, lol).

How did you find the school?

At some point when I started playing, I figured it would be good to get direct feedback once in a while. So I looked for instructors online. And you see, when you look for english-speaking baduk teachers on Google, there is a limited number of search results and it doesn’t take long to find all of them.

What was the occasion to take a whole month to study?

I’m basically a dinosaur fossil at this point.

But seriously, my employer (Figma) has a sabbatical program where you get to take an extended leave after you’ve been there for 5 years (this is considered a long time in tech industry where the average tenure is two years). I used half of it for baduk.

Did you know baduk already?

Yes, I’m not an expert but not a beginner either. I had a lot of free time to play/study during the pandemic before we got vaccinated and there was nothing else to do. I had been going to the Berkeley and San Francisco go clubs on occasion. I’m about 3 kyu — for those of you who don’t know what that means, it’s something like “mid-advanced intermediate”. I was still the weakest player at BIBA, but your strength doesn’t really matter. There’s no entry tests to get into BIBA, you just email them and ask if you can come. The school seeks to promote baduk among foreigners, not produce professionals.

What was a day like?

It depends on the day.

On many (perhaps even a majority?) of days, there was something on the schedule. Examples:

Going to the Hanguk Kiwon (Korea Baduk Association/KBA), which is also the TV set where they play and televise official matches like you might watch on Baduk TV. The teachers are professional players themselves, so they have official matches around once a week. While the teacher is playing, we are in the backroom reviewing the game.
Going to academies where kids (in some cases, yunguseng) where they find kids our level to play against. Afterwards, the teacher review the games.
Similarly, arranging team matches with players at clubs for middle-age/older Korean guys.

KBA back room
Academy of aspiring pros
BIBA students v.s. Korea amateur players

On other days, it’s generally studying on your own and playing games with the other students. The teachers will provide problem sets on demand and correct the answers, but the amount and intensity of studying is largely up to you.

A typical day didn’t start until early afternoon — early morning was a good time to self-study. In the evening, it was typically board games (the Catan type) or eating/drinking/poker in Seoul.

An evening at home

KPMC

The Korea Prime Minister Cup, a world amateur tournament was held while I was there which was timely.

http://kpmc.kbaduk.or.kr/eng/

I was not my country’s representative (Canada has too many strong players), but I got to sneak in since the teachers were working at the tournament as english-speaking professional commentators/game reviewers. This way, I got the chance to meet a lot of top amateurs and play against the representatives of some countries that were around my level in the after-hours.

During the tournament -- if you look closely, you'll see Yoonyoung working as the head referee next to Lee Yujin

Funnily, I managed to get myself a “job” while there. They needed some extra people who understand the game to record the games while they were being played for the purpose of livestreaming to the the Internet, so here’s me with an official vest, and a link to a game I recorded, Belgium v.s. New Zealand.

Who were the other students?

Most of the other students this year were their country representatives for KPMC, and wanted to stay a few extra weeks in Korea before/after the tournament. There was only me and a Russian teenager who were not there for KPMC. The party grew quite large, at peak I think we were around 8 students. In terms of age, the mode was around 25-30 with the youngest being 14 and the oldest being in their 60s.

Funnily, there was a significant French-speaking contingent there, between myself (from Quebec), the Canadian rep (also from Quebec), the French rep, the Belgian rep, a former BIBA student (also from Quebec) and some friends of BIBA (from France) who occasionally showed up so we spoke French a lot.

What are some highlights?

The best part about being part of a baduk school and being there around KPMC is that I got to meet a lot of professional and top amateur players. There are only about 400 or so baduk professionals in Korea, most of whom have been studying in baduk schools since a young age. It’s a close-knit community, kind of like how tech people befriend other tech people but even moreso. As such, we met a lot of other pros via the social circles of the teachers and went drinking/bowling/etc.

This gives a glimpse into the life of a baduk professional, not as those monstrously skilled players, but as humans with aspirations (such as travelling, getting better at english) and who effectively work as freelancers. Being a baduk pro in Korea is not really a job so much as a title. It gives you the ability to play in tournaments (and be paid), but otherwise it’s up to the individual to figure out what to do with it.

Other than that, some notable moments involving recognizable names:

I got to play against Choi Cheolhan, who was the former #2 in Korea during the time of Lee Sedol. From my understanding, he was just being a dad that day and bringing his kid outside. But his son was the strongest 5-year old in Korea, and was actually about my level, so the teachers had us play against each other. I won, but I will almost certainly get crushed if I play him again in a few months. I guess that earned me a free a teaching game against Choi Cheolhan, which was quite lucky! Normally, it’s hard to get a game against the more famous players (or at least, it’s quite expensive).
At KPMC, there was a pair-go side tournament that anyone could play in (including the pros, referees, etc). I played with Yoonyoung Kim, one of the few english-speaking baduk content producers, against Javier from Finland and Kenyi from Peru.
Robert van Zeijst, the Dutch representative and author of many Go books hung out with us at BIBA after the tournament.
Being around the KBA a lot, we saw notable people just pass by, like Lee Changho (former #1, ~1990-2005), Choi Jeong (current female #1), Byun Sangil (current #2). We didn’t see Shin Jinseo, but I did get a face mask and a cup with his face on it.
Lunch with Haewon Han and Eunkyo Do and learned more about their backgrounds as baduk pros.

Me v.s. Choi Cheolhan's son and simul with Choi Cheolhan
A mask to give you power

Was it what you expected?

Unlike many of the other students who learned about this school from word-of-mouth, I went in completely blind, so I didn’t have very set expectations. That being said,

It was a more immersive than I expected and I didn’t think that I’d get to meet so many new people throughout the professional & amateur community
The days were a little bit less structured than I expected
But we went out to visit a lot more places than I expected
There was not much touristing Seoul (which is fine, I’ve visited before)

Overall, it was a great experience! The opportunity to meet so many people was inspiring.

Did you become stronger?

This one is hard to say. The dan players at the SF Go Club said my opening has improved, and I certainly increased my “vocabulary” of moves while at BIBA. On the other hand, I didn’t play online much while at BIBA or after (urgh, circumstances, I’ve been very busy after coming back to the US) so it’s hard to say if my rank changed.

Like most foreigners, my weakest area is raw reading ability which can only be fixed with large volumes of problem sets, which implies heads down in the library studying. However, going to BIBA certainly increased my motivation to do so, and to be more involved with the go community at large.

Can I go to BIBA too?

Just email them! Though they are no longer open year-round like they used to, Blackie and Dia have expressed interest in opening it again next year.

They also teach online and can connect you with young upcoming Korean pros.

GraphQL, meet LiveGraph - a real-time data system at scale

Thu, 14 Oct 2021 12:00:01 +0000

If you are not redirected automatically, follow this link.

A case study on strict null checks

Thu, 17 Dec 2020 12:00:02 +0000

If you are not redirected automatically, follow this link.

The hidden challenges of autosave

Tue, 27 Oct 2020 12:00:02 +0000

If you are not redirected automatically, follow this link.

How to build a plugin system on the web and also sleep well at night

Thu, 22 Aug 2019 12:00:02 +0000

If you are not redirected automatically, follow this link.

Learned at Figma - what makes a good engineer?

Sat, 29 Sep 2018 12:00:02 +0000

In my previous blog post, I talked about my learnings of the properties of Figma as a company. This second part will focus more specifically on things that I learned about engineering as a result of being at Figma.

Knowing what to work on

In school, we’re taught tools to solve problems. During internships, we’re given a chance to practice using those tools. And this is as it should be — you need to learn basic addition before learning algebra.

But what I came to learn is that the further out from school you go, the less you are provided with clear directions. The narrow path widens into an open field. So the skill that becomes increasingly relevant is not just “doing well something someone asked you to do”, but “figuring out what the thing to do is to begin with”. Fortunately, I’ve had plenty of chances to exercise this skill at Figma and watch other people do so.

On a small scale, this means asking questions like:

Does completing this task help my teammates? (e.g. unblocking or refactoring)
Does completing this task help us figure out uncertainties? (e.g. tell us something about whether our approach on a product or technical problem will work)
Does completing this task help plan the project better? (e.g. task that are at risk of taking a long time are better done early than right before a launch)

And on a larger scale, it means asking similar questions but being even more strategic.

One of my team’s big projects took 6 months, when very initially, we planned for a little over a month. Why were we off by a factor of almost 6? Part of it was just that we didn’t quite have in mind the multiplicative complexity aspect of Figma and didn’t plan for a lot of the small features we’d have to interact with. But what really took the most time was that 6 months was really three attempts that each took 2 months.

The problem was that we originally came up with some product designs, which people deliberated on and one was “approved”. We started working on this, and ran into a lot of interesting technical challenges. Product challenges also came up as we found more designer use cases that we wanted to support without having too many ad-hoc options. In the process, we often deliberated and iterated on those details. All reasonable things. But when we did user testing, the high-level model turned out to be confusing to users and ultimately had to be thrown away.

There was nothing wrong with trying different approaches, of course. It’s a natural part of the product process to iterate until you create something good. The mistake was that we prioritized the difficult technical challenges without realizing that the main area of uncertainty was the product. Since then, I’ve advocated for quickly prototyping a basic design with the aim of producing something user-testable as fast as possible. It turns out that once you’re familiar with the codebase, if you ignore interaction with other features, good engineering practices and writing test, you can build a prototype of a feature in like a week. Sure, this might involve very dirty hacks like copy-pasting entire files to avoid spending time refactoring and other unacceptable things to do on the main branch. But those good practices can be done once you’re sure you’re working on the right problem.

Teamwork

One particular aspect of knowing what to work on is knowing how it impacts the team around you.

At a very basic level, this means things like:

Work on tasks that will unblock your teammates first
Refactor code that you think could cause confusion to your teammates

A little bit more advanced would be things like:

Write a new testing infrastructure that makes it faster and less tedious to write tests (real example, it’s called interaction tests at Figma and it’s great)
Replace a pattern of using raw pointers with weak pointers to make sure no one accidentally runs into undefined behavior (also a real example)

But those things are pretty self-explanatory in how they help. It takes some skill and experience to learn to recognize them, but it’s not too hard. Harder is figuring out your impact as an individual agent in a complex system made up of many humans. A driver changing lane abruptly has no awareness of the traffic wave that propagates miles backwards.

One way to think about planning work efficiently in a company is that you have a highly non-linear, multi-dimensional function to optimize. Variables include things like what the tasks are (not always all known ahead of time), the time it takes to do each task (not always known ahead of time), the dependencies between the tasks (not always known ahead of time), the skills of available engineers, the desires of available engineers/what they want to work on, random stuff like urgent bugs or people going on vacation, etc.

As you’d expect, optimizing such a function is really hard, especially since the function contains lots of unknowns. So intuitively, you can imagine that the more you can reduce those unknowns, the more easily the function can be optimized. Even if it’s not my job to optimize that function (it’s my manager’s and his manager), it certainly help the company if everybody pitches in a little bit.

Part of it is good communication. One of our engineering values is “communicate early and often”. Communication reduces unknowns which generally helps everybody.

But the big one I learned is that it’s important to be predictable in how long it takes me to do stuff. Up until now, I’ve always had an aversion towards making time estimates. And it is true that it’s not always feasible, in which case forcing an estimate is worse than not having one. But it turns out that estimating is yet another skill among many, and the more you do it, the better you get at it. The better you get at it, the less painful it becomes.

Another aspect of being predictable is to sequence things appropriately. I often ran into situations where I saw opportunities for improvements to the codebase. But if I’d gone down and followed it right away, it would have added a couple days to do tasks and that would probably cause some traffic congestion elsewhere. So I made note of those things I wanted to improve and during down times when there was less inter-dependencies, I actually made those improvements happen.

How far technical skills get you

In these last few months, I learned skills about product engineering as a full-time engineer that I hadn’t had the chance to exercise as much as an intern. And if you squint at them a little bit, a lot of these skills are what you might call soft skills. The traditional advice isn’t wrong: soft skills are important.

But I find that a lot of the time, this advice is framed in a context as exclusive to raw technical abilities, such as being a fast coder. No one encourages you to practice competitive programming after school, presumably because there are more ‘important’ things to improve on.

What I’m finding from working (i.e. observing other people) at Figma is that raw technical ability seems to unlock more than increased production. It makes it possible to do things that in practice just don’t ever get done if they’re anticipated to take too long.

Consider, for example, building a prototype for a feature. If the feature is fundamentally very complicated product wise, it shouldn’t matter if it takes 2 days or 2 weeks to build it as long as that is the optimal thing to do in terms of expected value \(E\). In practice, it’s hard to feel comfortable spending weeks building prototypes due to the worry of wasting time on a failed experiment, so that step easily gets skipped over.

There’s other types of work, such as refactoring some core internal API or abstractions, that interfere a lot with ongoing work by other members of the team. An extreme example, for example, would be to rewrite the application in another language. This has to be able to get done fairly quickly, otherwise it just causes too much disruption for it to be ever worth it (or be perceived to be worth it).

Found this blog post interesting? Please leave a comment! Remember to check out figma.com/careers and feel free to ask any questions!

Learned at Figma - design tools and multiplicative complexity

Sat, 29 Sep 2018 12:00:01 +0000

Figma is my first job out of college and I’ve been there for a little over a year now. It’s a good time to write about my experience and stuff I learned, as I’ve done in the past in internships.

Now, not to spoil too much, but I’ve liked working at Figma so far and I’m still there. So no surprises if, to some extent, I’m hoping you’ll get interested in Figma too. We’re hiring 😉! But as always, my priority is first to try to write something that you’ll find insightful. As I found out, Figma (the application) has a unique combination with properties of both a web app and a game engine. That leads to a lot of fun challenges.

Things I’ll cover:

Design tools turn out to be quite interesting
Startup life at Figma
Multiplicative complexity
Experiments and side projects
The culture at Figma

What is Figma?

First and foremost, it’s worth setting a little bit of context. Figma is a startup located in San Francisco, with 70-80 employees total as of writing. We build a design tool for creating user interfaces. Think Photoshop/Illustrator/Sketch. And it’s collaborative (e.g. multiple people can edit at the same time) as it runs in the browser. Think Google Docs. The gist of our business model is to offer a free product to individual and charge teams for advanced collaboration features.

Most of my time has been spent working on prototyping features — that is, features that allow designers to build prototypes of their design by adding interactions in their otherwise static design.

Design tools turn out to be quite interesting

I didn’t know much about design tools when joining Figma. The extent to which it mattered to me is that design tools are interactive content creation tools, and I’ve always liked playing with content creation tools (blender, flash, particle illusion, etc). As a programmer, I also perceive them as being more fun to work on than page-based applications (e.g. most websites). Otherwise, my main attraction to Figma is that they used WebGL and they were being creative in finding ways to run an app on the web that was traditionally in the domain of desktop software. But I decided to let the design space be a known unknown: something I’d figure out once I’m there.

Now that I’ve gotten acquainted with this type of design, I’ve actually been enjoying it a lot. Previously, the word ‘design’ in my head was heavily associated with the word visual. How do you make a webpage look good? But in practice, figuring out the look of the UI is only a small part of the process. There are bigger questions to answer like:

What are the top use cases for our customers?
How do we build features that form a consistent system instead of individual ad-hoc features?
How do we build features that are easy to understand and will do the right thing by default?

So really, these problems become appealing as they are less art problems but rather more psychology problems. No wonder that the famous ‘The Design of Everything Things’ reads more like a psychology 101 book than anything else.

Beyond that, designing a design tool has a lot of different properties than consumer apps. For instance, Figma isn’t something our users use occasionally or passively. Our users are professional designers that often spend multiple hours of their day inside the tool. As a result, they want a lot of things and can be quite opinionated about it. But it also makes it all the more satisfying when we release a new feature and we can see the immediate improvement in their workflow and their direct appreciation.

Indeed, one pleasant surprise is that we interact pretty directly with the customers. While we have long term visions for our product, we’re quite customer-driven in terms of how we prioritize features.

The designer ↔ engineer collaboration aspect is also really fun. Another interesting property of design tools is that it’s usually not as simple as having the designer write up a spec, draw up some UI, and then hand it off to the engineer for implementation. For certain applications, you can often do that because the extent of the interactions with the website is a couple of places to click and a few forms to fill.

But for design tools, you have a highly interactive canvas where the user can assemble all the lego pieces you’ve given them in ways that can get fairly complex. Many variables can affect the experience, such as the number of actions needed to achieve a task, whether those actions are allowed to happen in the order you might intuitively want them to happen, etc. There’s no way to know for sure how good a feature will be until you’ve had the chance to play with it. Furthermore, the complexity of some of the features means there are a lot of cool ideas that are not realistic to implement in a given timeframe, because of one technical restriction or another. All this means that there’s a lot of back-and-forth between engineers and designers which makes the whole process more fun.

Even as I did product work during some of my internships, I never really identified as a product person. I liked reading papers & stuff like compilers. But I’m learning from working at Figma that I am more of a product-oriented than I thought, in large part because the product problems here are fundamentally very interesting.

Startup life at Figma

Usually, when people choose to work at startups, there are recurring reasons for why. You probably heard them: more responsibilities, faster career growth, etc.

I’ve gotten some of that. Previously, during internships, projects were well-scoped projects even when they were technically involved. At Figma, it starts with ‘soooo we’ve been getting a lot of requests about feature X, Y & Z lately, what do they mean?’. And along with the rest of the team, I generally have at least a hand in every step from that leading up to the launch: doing some user research, figuring out which of XYZ is most important, writing up proposals, making estimates, implementation, coordinating with marketing and support, etc.

Having more responsibilities at a startup has been good for growth. While these are certainly not exclusive to startups, my naive new grad self would like to think it’s given me more opportunities to do more things.

Among some things I didn’t anticipate but found fun has been working with the support team to figure out customer problems or sometimes directly providing some support to customers. As part of creating new features, we also invite some of our engaged users to participate in user tests and it’s been fun seeing how they go about using Figma.

Another appealing premise that has held true is that at Figma, you can succeed by pushing the team forward (whereas at larger companies, you have to manage your own career more). I think this is made possible by the fact that the engineering team is still small enough that you can know more or less everyone and what they’re up to. There are career and growth guidelines, but no formal title or performance review system. Presumably we’ll need to have some eventually as we keep growing — they do exist for a reason — but for now I’m content with how things have been.

Lastly, I often get asked about work hours. They’re quite normal. I usually get in anywhere between 9:00 and 10:30, and leave around 6:00-6:30. As is culturally common in tech, people do engage in Slack outside work hours sometimes but I don’t see instances where replies are expected (except when the service is down of course). There’s consideration for work-life balance at Figma, and enough experienced engineers that the work 80h/week startup phase got skipped over entirely as far as I can tell.

Multiplicative complexity

Back to how it’s like to work on a design tool.

Unlike most applications where different features are neatly separated into different pages, everything in Figma happens in the same space, the canvas. So whenever you intend to add a new type of building block, you might figure out how it’s going to interact with every other building block.

Thus, adding features to Figma seems to have multiplicative complexity as opposed to additive complexity. And I believe it’s an inherent property of the domain space, not a failure of us building poor abstractions. Every time we add a new feature, there are real product decisions to be made when it comes to the interaction with existing features.

For the engineer, this obviously makes the application fun to work on, as even simple features come with fun challenges. But another consequence of this is that since complexity can grow very fast, it needs to be managed carefully. As such, for a startup, I found that people invest a decent amount on the codebase infrastructure and refactor regularly to keep the complexity manageable. It doesn’t mean of course that we have the luxury of spending endless time on technical perfection — every refactor is a project with costs and benefits that need to be evaluated. But the key is that the incentives are there to maintain good engineering practices.

Another interesting thing I learned about Figma that isn’t apparent at first glance is that it has both the properties of a web app and a game engine. It has the properties of web application in that it needs to handle scale, both in terms of number of users and the size of the documents the users create. It also has the properties of a game engine where lots of different components have to interact in the same space and do it in real-time, i.e. at 60fps. Combined, this means there are optimization problems of every flavor: scaling problems, CPU usage problems, memory usage problems, latency optimization problems, etc.

Those milliseconds make a real more-than-milli difference

So work is fun, and I expect work to keep being fun for the foreseable future because those properties are inherent to our product, not just the project I happened to be assigned to. Of course, I could be wrong that our problems are hard, which would put me (us) at a competitive disadvantage. But if you stay in the design tool space for a while, you’ll hear of new design tools being announced every other week that each promises a lot. The rate at which they get delivered is slower, and my guess is that they run into similar challenges we do. 2D design sounds easy (common, how hard can it be compared to all the ✨ 3D tools out there?) but everyone eventually comes to this discovery.

Experiments and side projects

A fun aspect of Figma that I came to understand over time is that there’s both a need and an opportunity for side projects and experimentation.

It’s common for engineers at Figma to spend a little bit of time outside of their main assignment to do various sorts of things. This has included things like writing docs, coming up with a set of eng values, cleaning up our interview process, improving developer ergonomics, trying out a new renderer, or building hacky prototypes of commonly requested features.

The first few are important as a startup grows. Even though not as urgent as shipping the next release, they build the foundation that will allow future structures to stand upon.

Building prototypes has been pretty interesting to me. We have a very long list of feature requests from our users. Many of them are very challenging, both product wise (many approaches) or technical wise. It’s difficult enough to figure out which ones would be the most impactful — it’s even harder to calculate the cost/benefit ratio since it can take days or weeks to understand the requirements well enough to calculate the cost. So there’s a lot of projects that float in the space of “we know it’s important, we’ll do it…at some point”.

When someone builds a quick prototype of these features, the effect is often one of a few things. It turns an abstract idea into a concrete demonstration that people can comment on or suggest improvements to. Demos inspire idea generation and action. It also makes previously big-looking projects seem more achievable, either by showing that the solution turned out to be simple, or that a partial solution would be more than sufficient to satisfy a lot of use cases. For ideas that involve large system changes, it allows validating that the new system would have the intended benefits before investing a large amount of time switching systems in the real codebase.

And we’re able to actually spend time on these 20% time type things, because while we do have deadlines and occasional sprints before launches (it’s important to get things done!), we don’t have them so often that there’s only time for the urgent stuff. It’s not an official policy and there are ways it can be improved (e.g. not everyone might feel comfortable doing side projects) but it’s an interesting balance.

Overall, I think this follows from the property that engineering at Figma is a little too complex for central planning and it’s necessary for individual actors to have agency. It’s not unbounded freedom as the success of the business is still my business. But there is freedom insofar as you have to trust the individuals that the things they think is most important from their local perspective is actually important, and give them the freedom to act on it.

The culture at Figma

One of the great things about working at Figma is it’s culture. But the idea of a company’s culture is a challenging concept to capture and Figma’s culture isn’t very colorful in the sense that it doesn’t have a very standout/distinctive trait like some other places (e.g. “be a host” or “know what you don’t know”).

If anything, it would be more that people at Figma on average are very thoughtful, and this translates into a lot of small things done well. Our code review process is your usual Github workflow, but people are pretty good about using it as a teaching tool rather than a gatekeeping tool. We use fairly standard programming languages (e.g. C++, Javascript, Ruby), but in a way that’s generally quite understandable and avoids major pitfalls. We don’t have an elaborate system of peer review and promotions, but there a lot of avenues like 1-1s to get feedback and achievements are given recognition publicly. We have a pretty standard format for interviews: 10 min of introduction/chatting about background, 45 minute of coding and 5 minutes of ask-me-questions, but we avoid asking unrealistic algorithm questions that characterize some of these types of interviews and let people code on their own laptops. Leadership cares about diversity, but by getting a lot of feedback internally and externally, not just tacking it onto the company values.

We also have a set of official company values and a set of engineering values. I won’t be doing them justice with a single paragraph in this blog post, but definitely ask us about it if you come visit 🙂.

Of course, when it comes to something like culture, nothing is ever perfect. As I’ve written before, every company has some boundary (often defined by the needs of the business) where tradeoffs needs to be made. As a simple example, we have large open lunch tables where people trickle in and out. It’s a great way to get the chance to interact with everyone at the company, but makes it difficult to go into depth in a topic as that requires building context and exclude the people who trickle in. But if thinking about these tradeoffs means we’re reasonably close to that boundary, then I think we’re in a good spot!

Thanks to Shirley Du, Andy Zhang, Jamie Wong, Susan Wang and Ryan Kaplan for reviewing this blog post!

Yellowstone - filming my first travel video

Sat, 04 Aug 2018 12:00:01 +0000

I filmed this travel video last month when I went to Yellowstone National Park. It was an interesting learning experience as I’d never done any filming or video editing before pen spinning aside.

This blog post is mainly about how that process went. My main two takeaways are 1) getting started is relatively easy and 2) filmmaking is a rich subject with endless things to learn and ways to have fun! While some of you might be into photography, I imagine that fewer of you have considered filmmaking. I didn’t until two months ago. So hopefully this can give you some ideas!

Getting started

First of all, how did I think of making a travel video? It was very much on a whim, really. I saw a gimbal at the Apple Store and thought it would be a fun toy to play with.

If you don’t know what a gimbal is, it’s a type of stabilizer that keeps the camera pointed in one direction. This gets rid of most of the shakiness from handheld footage. The one I got is part of the newer class of consumer models where you can even stick a smartphone on it. So that gets you instantly 80% of the way to making smooth, cinematic footage (smooth enough that software stabilization can handle the rest).

So there we had it: my equipment was going to be an iPhone X and a DJI Osmo 2 gimbal. That’s pretty much all I used, except is a few places like the bear scene where I also had a telephoto moment lens.

That was less than a week before the trip, so I promptly looked up how to get that cinematic look. As with many things, YouTube comes to the rescue.

YouTube is really like its parent Google these days, where you can find just about anything. In any case, as you can see, there’s many videos with many tips. Too many to focus on at once. In the spirit of picking a subskill and having something to focus on, I felt that the highest leverage tips were 1) always move the camera and 2) don’t shoot at eye level.

Without camera movement (even simple pans, which is most of my video), the ‘video’ feels more like a slideshow than a film. Movement is what creates the cinematic feel. And it’s important to get interesting viewpoints – eye level pictures/videos tend to look like everyday tourist pictures. It boils down to being creative about finding varied angles or interesting close-up subjects.

There’s many other important elements to a good video for example, such as telling a good story and having good camera coverage. But those require some amount of planning which means you need something to plan for — a vision of what you want the video to be. But I figured it’s too early for me to have that sort of vision, which is why I focused on the more ‘technical’ aspects like movement.

Filming at the park

I don’t have too much to say about the actual filming process while I was in the park. Having watched some tutorials, the next step was mainly to get comfortable with the equipment, trying out a variety of movements/settings to get a feel of what gives good looking results. i.e., the practice part of theory & practice. One big difference in shooting for a cinematic video (v.s. just recording videos on my phone) is the mindset that most shots will be at most 1-4 seconds long in the final video, so it’s almost more like a sort of ‘motion photography’.

We stayed in West Yellowstone where we had wifi. So after each day at the park, I had time to watch a few travel videos on YouTube.

This is helpful to get a sense of all the different people’s styles and techniques they use, and get some ideas for shots to film. So I learned a bunch as the trip went along — I stopped shooting in 24fps and used 60fps as the default for example (more on that later). As mentioned in the beginning, filming for this trip was a very impromptu decision and I knew nothing about travel filming — I didn’t even know about Sam Kolder for instance (apparently, his videos have been influential to the point where having a Sam Kolder style video is a cliche).

Back home and editing

Similarly on the post-processing and video editing side, I had no experience. The closest was adjusting contrast and clarity in Lightroom. So first step was to figure out what editing software to use. I initially wanted to use my Windows desktop since it has decent specs (in which case the safe choice is Premiere). But I wasted a day on the iPhone not cooperating with Microsoft & Adobe, so I gave up and went for Final Cut Pro X (FCPX), on my work laptop.

Filtering footage is straightforward enough — make a shortlist of takes with interesting enough content and find 3-4 seconds of usable footage within them (a segment with continuous camera motion/no jerking).

The choice of music dictates a lot about the length, tone and structure of the video so I had to find a song first. To avoid dealing with copyright issues (especially for the scope of a project like this), I found a convenient channel with free music

Choosing the right soundtrack was challenging — there are so many! And again, I didn’t have enough experience to visualize what an edit with each soundtrack could look like. I got myself unstuck when I realized that I could play the music while scrubbing random shots in FCPX.

Putting together the video wasn’t too hard. There’s a lot of fancy stuff that can be done with transitions and interleaving different shots to set the mood, but my footage didn’t really lend itself to doing anything too fancy anyway (fancy transitions often require you to film with the transition in mind, which I didn’t). So I went with only straight cuts or fades.

The most difficult part was color grading, the process of mapping the colors in your video onto a particular palette to give the film an overall consistent look. See this super cool article on color grading in films for instance. I wanted to do some color grading since it won’t look as cinematic if it looks like any other iPhone video.

One way to do that was to apply a LUT (3D lookup table), which is basically a filter but for videos. I tried a whole bunch and none could make one shot look good without messing something in another. For example, if it made the landscape amazing in one shot, it’d mess up skin tones in another. In the end, I applied a light amount of a fairly simple LUT and added a tiny bit of teal & orange. The result’s fine, but I’m less satisfied with the color grading as with other parts of the process.

For next time

In the process, I learned how to use my equipment better. Initially, I was shooting at 24fps by default but switched to 60fps halfway in. The original rationale was that 24fps is the frame rate in movies and gives videos a more cinematic look. The problem was that didn’t actually work in my case. The cinematic look comes from increased motion blur, but you can only get motion blur if the shutter is open long enough, which it almost never was since I was mostly shooting in bright daylight. In the future, I’ll have to get an ND filter to use 24fps. Otherwise, 60fps is nice since most shots look more cinematic slowed down without screaming SLOW MOTION the way 120fps does.

I never got the tracking feature on the gimbal to work, but I also found halfway in that manual tracking wasn’t that hard, so I’ll probably do more of that.

Now that I’ve acquainted myself with some of the basics, things to do in the future include:

Working on having more coverage
Planning for more advanced transitions
Shooting with the intended color grading in mind
And more!

Thanks for reading!

Is implementing Operational Transform hard?

Thu, 04 Jan 2018 12:00:01 +0000

Operational Transform (OT) is a class of algorithms used in tools like Google Docs or Dropbox Paper to support collaborative editing, where multiple people work on the same document at the same time. It’s also reputed to be quite complex. There’s a famous quote by Joseph Gentle, one of the early people to have implemented OT, that shows up on Wikipedia and in nearly every blog post that talks about OT:

Unfortunately, implementing OT sucks. There’s a million algorithms with different tradeoffs, mostly trapped in academic papers. The algorithms are really hard and time consuming to implement correctly. […] Wave took 2 years to write and if we rewrote it today, it would take almost as long to write a second time.

However, what is less known is that the same Joseph Gentle later retracted that statement in two Hacker News comments:

Yes thats me! For what its worth, I no longer believe that wave would take 2 years to implement now - mostly because of advances in web frameworks and web browsers. When wave was written we didn’t have websockets, IE9 was quite a new browser. We’ve come a really long way in the last few years.

and

You’re right about OT - it gets crazy complicated if you implement it in a distributed fashion. But implementing it in a centralized fashion is actually not so bad. Its the perfect choice for google docs. Here is my implementation of OT for plain text: https://github.com/ottypes/text Note that its only 400 lines of javascript, with liberal comments. To actually use OT code like that, you need to do a little bookkeeping. Its nowhere near as bad as you suggest.

But as it often happens, the quote took a life of its own while the retraction is buried in the archives of Hacker News comments.

I bring this up since the quote is getting relevant again now that CRDTs are arriving into the scene. CRDTs are a newer class of conflict resolution algorithm. Thus, people will inevitably start comparing OTs and CRDTs to understand the differences and decide which of two to implement.

So what’s the conclusion? I’ve never personally implemented an OT system, so I can’t speak from first-hand experience. But I’m acquainted with a few people who’ve tried to implement various flavors of OT. My understanding so far is that:

It takes effort to understand OT and makes for a fun and challenging side project
OT with a central server is not too hard to implement, especially for plain text OT
OT with a central server has been implemented enough times that there’s a lot of resources and knowledge floating around
In those cases, the code for OT won’t be too complicated either
Fully distributed OT and adding rich text operations are very hard, and that’s why there’s a million papers
The most difficult part of OT is not the code, but the difficulty in proving that your system is correct. Therefore, maintaining OT code is difficult. Either you need to prove your code correct repeatedly (and historically people make mistakes), or you need a powerful testing infrastructure for concurrent/distributed systems (which is also hard to write)

A simple approach to building a real-time collaborative text editor

Fri, 06 Oct 2017 12:00:01 +0000

This post is my attempt to write an easy to understand introduction to the main ideas behind building a real-time collaborative text editor, which can be tricky as you want all clients to synchronize and see a sensible result even in the face of concurrent edits.

In some ways, this is already a solved problem by Operational Transform (OT) and open-source implementations like Firepad and ShareJS. So if you want a plug-in-play solution, use one of those and move along. I decided to use a CRDT (Conflict-Free Replicated Data Type), a newer technique (more specifically, I adapted Logoot). There are technical pros and cons versus OTs that I’m not gonna bore you with, but for me the major advantage of the CRDT approach is that it’s simpler to understand and to convince yourself that it works (and thus, to maintain it).

I hope to demonstrate that in this post by splitting the problem of implementing a CRDT into multiple conceptual layers. Those layers should be fairly easy to understand one at a time. In contrast, OTs work by requiring you to handle every combination of operations arriving in different orders (i.e. all the weird shit users do) and proving that you’ve handled all of the edge cases. While the code itself may not be that complicated, if you make a mistake, you’re screwed.

This was a fun side project to build while at Recurse Center, and also quite relevant now that applications are increasingly moving to the web and becoming collaborative. Good ol’ Google Docs showed that it’s quite useful for multiple people to be able to edit at the same time. Now, Dropbox Paper, Overleaf, Coderpad, etc all support collaborative writing.

The problem of synchronization

First, I’ll briefly introduce what makes collaborative text editor hard. Let’s say you have two students, Ping and Pong working together on the same essay, and they’re working on different parts of the document. Ping makes a change: insert the string “, sadface” at the end of the line. The naive way in which Ping’s editor could transmit the change to Pong’s editor is just to send a message insert(", sadface", col=44).

And this would work if changes could be transmitted and applied instantaneously (possibly breaking causality along the way).

In practice, Ping’s neighbor is torrenting The Emoji Movie (don’t ask me why) and causing Ping’s messages to arrive really slowly to Pong. While Ping’s message was stuck in emoji traffic, Pong added the string “, not” at column 23.

Wops. By the time insert(", sadface", col=44) arrives, the end of the line is no longer column 44. We still insert though, and get this:

Now, we’re in an inconsistent state and The Matrix falls apart around Ping and Pong.

Commute + idempotent => synchronize

Let’s look at the first insight that’ll allow us to make a collaborative text editor.

In the previous example, two operations were generated concurrently on different clients. When we send them over to the other client, they end up applied in different orders and the clients end up in different states. They don’t commute.

On the other hand, if every operation could commute with every other operation, the clients would converge to the same state regardless of which messages arrived where first. You still need to require that operations commute in a way that makes sense to the user of course. Lest you get a lazy programmer that makes every operation delete the whole file. “Hey, the states were the same in the end!” 🙄. But otherwise, if operations commute, synchronization is trivially easy. A basic example is an add-only set. It doesn’t matter the order in which you add things to the set, the set is the same in the end.

While commutativity captures 90% of the idea, in practice we need another condition for synchronization, which is that operations need to be idempotent. That is, you can apply the operation twice and it’ll give you the same result as applying it once. This is useful not only if your messages get duplicated somehow (maybe you’re using UDP instead of TCP? I dunno) but also if two clients decide to do the same operation at the same time. Then, receiving the other client’s (now duplicate) operation should be a no-op (do nothing).

I’m using the term “commute” loosely here. In practice you don’t need your operations to commute in the sense that EVERY operation commutes with EVERY other operation. It’s great if they do since commutativity is a sufficient condition for convergence, but it’s not a necessary condition. For instance, it doesn’t make sense to reorder operations generated by one client because you might end up putting DELETE(X) before CREATE(X). What we really want is some sort of “interleave-ability” defined by the causal order between operations and have the merging obey some partial order. But the goal of this post is to provide an intuition, even if at the expense of correct technicality. I’ll refer you to the CRDT papers for the big words.

Make everything (all kiwis 🥝) unique

At a very abstract level, a CRDT is just a data structure and a set of operations on the data structure that commute with each other. There’s lots of different examples of CRDTs. Some quite simple that you can read about while having breakfast. Others, well, less so.

Before we get into text editing, which is one of the less simple CRDTs, let’s create a simple CRDT to help us gain some intuition. We’re going to build a bag, also known a multiset. Like the name implies, it’s a collection in which you can put and retrieve objects. It’s like a Set, but it can contain multiple copies of the same element.

The two main operations on a Bag that we’re concerned with are:

Add(x)
Remove(x)

You probably also want to iterate over the Bag, count elements, etc, but those do not modify the Bag therefore don’t need synchronization.

Add and remove are not commutative. Imagine that Ping, Pong and Pang all have an instance of a bag that contains a kiwi (i.e. 🥝).

Simultaneously, Ping puts in another kiwi, Pong removes the kiwi and Pang removes a kiwi. What happens?

The fact that both Pong and Pang both decided to remove a Kiwi from the bag at once is not a problem (although jeez, Pong, Pang, why did you have to do that?). Remember, operations are idempotent and the end result should just be that the kiwi got removed from the bag. The problem is that Ping has two kiwis in his bag (🥝🥝) when receiving the two DELETE(🥝) messages from Pong and Pang so Ping deletes both kiwis 🥝🥝. In the meantime, Pong and Pang will end up with either one kiwi 🥝 or no kiwi 🥝 depending on whether ADD(🥝) or DELETE(🥝) arrived first.

It’s The Kiwipocalypse 💥🥝🔥.

We can fix this by changing the semantics of the operation a little. Now, not only are kiwis 🥝 a unique fruit different from other fruits 🍓🍌🍏, but we’ll make every kiwi unique. I’m calling this The Kiwi Manifesto 🥝. More specifically, we’ll assign each individual fruit a unique ID when created.

The programmer will modify the Bag in the same way with ADD(🥝) and REMOVE(🥝). But the operation that will get sent to other clients will be ADD(🥝, <unique id>) and REMOVE(<unique id>). When adding a fruit to the bag, a globally unique id that has never been used before (in any client, ever) gets created and assigned to the fruit. When removing a fruit from the bag, the remove operation will remove the fruit from the local bag, get the removed fruit’s unique id, and send a message to other clients to remove an item in the bag with that unique id. Not exposing the unique id to the user also makes sure that they can’t ask to delete a kiwi that doesn’t exist yet.

This should make intuitive sense. Now, the delete message isn’t just about deleting a kiwi, it’s about deleting that kiwi. And you no longer run into ordering problems.

Finally, how do you generate new, globally unique ids? ~~You need consensus between clients, so implement Paxos.~~ Just kidding. Assuming that all clients have a unique identifier, make it part of the new object’s unique id. This makes sure that objects created in different clients are distinguishable without needing them to communicate with each other. To make sure that objects created by the same client are distinguishable, create a counter on each client that increments the bag is modified and add it to the unique id. In summary each object is tagged with (client id, counter).

Text editing: order in the fruit kingdom

Alright, but you say, “that’s cool, but I didn’t come here to build fruit baskets, I want to make a text editor”. Fine, fine.

A text document is not that different from a Bag (a bag of Alpha-Bit cereals anybody?). There’s a bunch of characters that you can add and delete. It’s just that the problem is a bit harder because each character has a position. That’s one way to look at it. Another way that’ll be more helpful to us is to say that a text document is an ordered bag of characters. The position of the characters then simply follow from that order (even if there’s newlines, those are just a “\n” character).

So what is the order going to be? What we’re going to try to do in this section and the next section is to design a way of generating unique ID for characters such that the sort order of those ID will tell us the order of those character in the document.

To do this, let’s first think about how we would insert characters in a line where all the existing characters have already been assigned unique IDs. Suppose our IDs were just integers, and we sorted characters by increasing ID. We might have something like this:

o c c u r e n  c  e
1 4 6 7 8 9 10 11 12

Wait that’s not how ~~occurence~~ occurrence is spelled. Spelling occurrence is hard ok? Anyway let’s fix that by adding an “r” between (“r”, 8) and (“e”, 9). But there’s no room! I guess we’re stuck with this spelling now.

Or not. A natural thing to do would just to create a fractional index between 8 and 9. If we insert the character (“r”, 8.5), then we get the expected result:

o c c u r r   e n  c  e
1 4 6 7 8 8.5 9 10 11 12

Therefore, we could use floats instead of integers to store our character IDs. In practice, we would want to store a list of integer representing the digits after the fraction so we don’t ever run out of precision. There’s also no need to have indices > 1 if we’re going to have fractional indices anyway, so we’ll just store the fractional digits.

Fractional indices by themselves of course are not globally unique IDs. Our IDs will need to be a bit more clever than that to be unique like snowflakes ❄️.

No two characters are the same

To generate globally unique IDs which I will now call position identifiers (since they identify stuff and position stuff), there’s two situations that we need to handle:

Two clients generating the same ID at the same time (e.g. insert a character at the same position concurrently)
One client generating an ID that was already generated (e.g. delete a character and reinsert it)

Here’s the high level idea on how we’re going to solve this. Assume all clients have a unique site ID. We’re still going to use fractional indices, but every time we insert a character and add or change a digit in a fraction, we’ll tag the digit with the site ID at which it was inserted.

This is how it looks like. Say that whale 🐋 starts a document with “my hero” and water buffalo 🐃 enters that same document.

m  [(1, 🐋)]
y  [(2, 🐋)]
_  [(3, 🐋)]
h  [(4, 🐋)]
e  [(5, 🐋)]
r  [(6, 🐋)]
o  [(7, 🐋)]

If water buffalo adds “super”, then the document might look like this:

m  [(1, 🐋)]
y  [(2, 🐋)]
_  [(3, 🐋)]
s  [(3, 🐋), (1, 🐃)]
u  [(3, 🐋), (2, 🐃)]
p  [(3, 🐋), (3, 🐃)]
e  [(3, 🐋), (4, 🐃)]
r  [(3, 🐋), (5, 🐃)]
h  [(4, 🐋)]
e  [(5, 🐋)]
r  [(6, 🐋)]
o  [(7, 🐋)]

If whale accidentally hits the keyboard and adds a bunch of “u”s (whales have fat fingers, I read it on Wikipedia), then it could look like this.

m  [(1, 🐋)]
y  [(2, 🐋)]
_  [(3, 🐋)]
s  [(3, 🐋), (1, 🐃)]
u  [(3, 🐋), (2, 🐃)]
u  [(3, 🐋), (2, 🐃), (1, 🐋)]
u  [(3, 🐋), (2, 🐃), (2, 🐋)]
p  [(3, 🐋), (3, 🐃)]
e  [(3, 🐋), (4, 🐃)]
r  [(3, 🐋), (5, 🐃)]
h  [(4, 🐋)]
e  [(5, 🐋)]
r  [(6, 🐋)]
o  [(7, 🐋)]

And so it went on, and a friendship between whales and water buffalos was formed!🐋!🐃! By making sure that the last digit of any newly added character’s position identifier is always tagged with the site ID, two clients can never conflict just by adding new characters.

You might think every position identifier could just contain a single site ID instead of a site ID for every digit, but you do need to tag every digit with a site ID. As we’ll see later, the sort order depends on the site ID so some site IDs need to be preserved to make sure that we’re inserting at the right place.

Finally, to handle the case of a client deleting and reinserting the same character (I know it’s you, silly monkey 🙈), every client gets a counter that always increments anytime an action happens. A simple integer counter suffices if you have a central server though the Logoot paper suggests using a Lamport Timestamp which is only two lines of arithmetic anyway.

Deleting characters is simply a DELETE(position identifier) operation, just like in the fruit bag example.

Now that we got:

Globally unique IDs with the CREATE operation
CREATE and DELETE commute
The sort order of the ID defines what the order of the character is to represent text

We got a collaborative, synchronized text editor! Well, mostly that’s true. I guess I should talk about some implementations details.

Show me ze money (ze code)

Up to this point, I’ve been handwaving over all the details to give you an intuition of how this CRDT works. But those details are kind of important to make sure we’ve covered all edge cases. I recommend also reading the Logoot paper for completeness. It’s a pretty readable paper. Note that I’ll be using similar, but slightly different terminology at places for ease of presentation. Let’s start with a proper definition of a position identifier.

A position identifier generated at site \( s \) is a tuple (position, \( clock_s \)) where the \( clock(s) \) is the Lamport clock value at site \( s \)
A position is a list of identifiers
An identifier is a tuple (digit, site id) where digit and site id are integers

The digits can be in any base you want. I personally use 256 and convert the number to a string for ease of debugging, but other reasonable choices include MAX_INT, Base64, Base85, etc. Note also that for simplicity, each position is between 0 and 1 insofar as the digits are concerned, so we can think of a position as \( p = 0.p_1p_2p_3… \) where \( p_i \) are identifiers and we don’t need to store anything left of the decimal.

In code, the data structures look like this:

class Identifier {
    digit: number
    site: number
}
class Char {
    position: Identifier[]
    lamport: number
    value: string
}

Next, we need to define the sort order between the position identifiers. Note that the clock value \( clock_s \) is not used for sorting since a client will never insert a character with the same position as another unless that character has already been deleted. So we’re really just defining the sort order between positions, which I implemented as such:

function comparePosition(p1: Identifier[], p2: Identifier[]): number {
    for (let i = 0; i < Math.min(p1.length, p2.length); i++) {
        const comp = Identifier.compare(p1[i], p2[i]);
        if (comp !== 0) {
            return comp;
        }
    }
    if (p1.length < p2.length) {
        return - 1;
    } else if (p1.length > p2.length) {
        return 1;
    } else {
        return 0;
    }
}
function compareIdentifier(i1: Identifier, i2: Identifier) {
    if (i1.digit < i2.digit) {
        return -1;
    } else if (i1.digit > i2.digit) {
        return 1;
    } else {
        if (i1.site < i2.site) {
            return -1;
        } else if (i1.site > i2.site) {
            return 1;
        } else {
            return 0;
        }
    }
}

This sort order is more or less what you’d expect, sorting by the digits and using sites as tiebreakers. You may refer to the paper for the formal definition and to make sure I didn’t make any mistakes. It’s worth making sure you really understand the sort order, because it affects how we insert characters. In particular, having sites as tiebreakers means you can have a sorted list of positions that looks like this:

[(1, 0)]                      0.1
[(1, 0), (4, 0)]              0.14
[(1, 0), (6, 0), (3, 1)]      0.163
[(1, 0), (7, 0)]              0.17
[(1, 1)]                      0.1
[(1, 1), (1, 1)]              0.11

Generating new position identifiers on insertion is more complex. I’ll describe what I came up with but be warned, while I made sure to write tests, this has not been tested in large production systems.

The algorithm described in Logoot is not suitable for our use case. In addition to missing what I think are key details on how to handle certain edge cases, the original paper was written for synchronizing individual lines, whereas a real-time collaborative editor needs to synchronize individual characters. Furthermore, when writing text documents, 99% of character insertions are going to happen on the right side of the last inserted character, with only occasional cursor jumps. We need to take this into account when creating position identifiers. Technically position identifiers are unbounded in size, but we can try to not make them grow too fast. If we were to naively, say, generate a position identifier halfway between two characters, the size of the position identifiers would grow very fast.

We can’t just do arithmetic on positions as arbitrary precision numbers because when two digits are the same, the sites could still be different. In the example above, we saw how this can lead to positions not being sorted in the same way numbers would be sorted.

Furthermore, as the “gap” in position between two characters gets smaller, we need to create smaller increments in position.

Because this algorithm is intricate, I’ll start with the code and explain how to think about it:

function generatePositionBetween(position1: Identifier.t[],
                                 position2: Identifier.t[],
                                 site: number): Identifier.t[] {
    // Get either the head of the position, or fallback to default value
    const head1 = head(position1) || Identifier.create(0, site);
    const head2 = head(position2) || Identifier.create(Decimal.BASE, site);
    if (head1.digit !== head2.digit) {
        // Case 1: Head digits are different
        const n1 = Decimal.fromIdentifierList(position1);
        const n2 = Decimal.fromIdentifierList(position2);
        const delta = Decimal.subtractGreaterThan(n2, n1);
        // Increment n1 by some amount less than delta
        const next = Decimal.increment(n1, delta);
        return Decimal.toIdentifierList(next, position1, position2, site);
    } else {
        if (head1.site < head2.site) {
            // Case 2: Head digits are the same, sites are different
            return cons(head1,
                generatePositionBetween(rest(position1), [], site));
        } else if (head1.site === head2.site) {
            // Case 3: Head digits and sites are the same
            return cons(head1,
                generatePositionBetween(rest(position1), rest(position2), site));
        } else {
            throw new Error("invalid site ordering");
        }
    }
}

generatePositionBetween is a function that recursively examines the most significant digits of an identifier by handling three different cases.

Case 1: Head digits are different

beforePosition = [(1, x), ...]
afterPosition  = [(2, y), ...]

This is the base case. If the most significant digits of the before and after positions differ, than it is possible to generate a new position such that beforePosition < newPosition < afterPosition regardless of the values of the sites x and y, simply by incrementing the digits of beforePosition by a small amount less than (afterPosition - beforePosition). Therefore, in this case, we first convert the positions to a decimal representation by dropping the sites.

// In module 'decimal'
export type t = number[]
function fromIdentifierList(identifiers: Identifier.t[]): t {
    return identifiers.map(ident => ident.digit);
}

The arithmetic operations add and subtractGreaterThan simply implement textbook addition and subtraction and you may refer to the source code for an implementation. The increment operation, on the other hand, is a bit more tricky.

function increment(n1: t, delta: t): t {
    const firstNonzeroDigit = delta.findIndex(x => x !== 0);
    const inc = delta.slice(0, firstNonzeroDigit).concat([0, 1]);
    const v1 = add(n1, inc);
    const v2 = v1[v1.length - 1] === 0 ? add(v1, inc) : v1;
    return v2;
}

All this does it increment a digit of n1 by 1 such that the increment is less than delta. Another tricky bit of increment is that we want to make sure that the result of increment does not have a 0 as the last digit. Why? Because then we would have to later make sure that we’re able to generate positions between two ambiguous representations of the same decimal (e.g. 0.201 and 0.2010). The simple trick used here is to increment again if that happens.

When we’re done generating a new decimal position, we convert it back into a position by assigning sites to it in such a way to maintain the sort order using toIdentifierList.

// In module 'decimal'
function toIdentifierList(n: t,
                          before: Identifier.t[],
                          after: Identifier.t[],
                          creationSite: number): Identifier.t[] {
    // Implements the constructPosition rules from the Logoot paper
    return n.map((digit, index) => {
        if (index === n.length - 1) {
            return Identifier.create(digit, creationSite);
        } else if (index < before.length && digit === before[index].digit) {
            return Identifier.create(digit, before[index].site);
        } else if (index < after.length && digit === after[index].digit) {
            return Identifier.create(digit, after[index].site);
        } else {
            return Identifier.create(digit, creationSite);
        }
    });
}

Case 2: Head digits are the same, sites are different

beforePosition = [(1, 1), ...]
afterPosition  = [(1, 3), ...]

Here, observe that newPosition = [(1,1), ...anything] will always satisfy newPosition < afterPosition.

More generally, if the site is used as a tiebreaker, notice that the new position could be anything that starts with the same identifier as beforePosition and will still be sorted before afterPosition. Which is why in the recursion

return cons(head1, generatePositionBetween(rest(position1), [], site));

We just pass an empty list as the second parameter.

Case 3: Head digits and sites are the same

beforePosition = [(1, 1), ...]
afterPosition  = [(1, 1), ...]

If the two digits are the same, then we need to look at the subsequent digits to find an in-between position, so we just recurse on the rest.

return cons(head1, generatePositionBetween(rest(position1), rest(position2), site));

With these three cases covered, we’re able to generate new positions for inserted characters.

Staying all in sync

To synchronize between clients, “add” and “remove” events are sent as such.

namespace RemoteChange {
    type t = ["add" | "remove", Char.t];
    function add(char: Char.t): t {
        return ["add", char];
    }
    function remove(char: Char.t): t {
        return ["remove", char];
    }
}

Here, Char.t is just an object containing a position identifier and a character. As in the case of fruit baskets, deleting a character just needs to delete by ID.

For simplicity, we always send remote changes as single character insertions or deletions. For efficiency, it would be better to batch them though that’s a simple optimization.

From abstractionland to userland

Let’s recap what we’ve created so far: a way of maintaining an ordered list of characters with unique identifiers and generate new identifiers in-between characters.

Using what we’ve seen previously, this is sufficient to represent a text document and synchronize it between clients. On the server-side, it’s actually sufficient to just store all the characters in an unordered hashtable, since the server only reads and relays ADD and DELETE operations.

Of course, on the client-side, users want more than a representation of a text document. Users want to be able to modify it with a simple interface. In my case, I am using CodeMirror as the underlying text editor, a popular open-source Javascript library.

CodeMirror has obviously no idea about CRDTs, and just stores the text content as lines of characters. This means I have to store two copies of the text: one in the text editor for client use, and one in the CRDT for synchronization use.

To interact with CodeMirror using the programming API, I can receive change events or send it patches using a EditorChange object that looks something like this:

interface EditorChange {
  from: {
    line: number
    column: number
  }
  to: {
    line: number
    column: number
  }
  added: string
  removed: string
}

(The actual object is a bit different but I changed it for ease of presentation)

That means we need to keep the CodeMirror editor and the CRDT in sync when any of the following four events happen:

Remote change: insertion
1. Get index of the preceding character in the CRDT
2. Insert character in the CRDT
3. Create a local change inserting a character at that index
Remote change: deletion
1. Get the index of a character with the same position identifier in the CRDT
2. Delete that character in the CRDT
3. Create a local change deleting a character at that index
Local change: insertion at position n
1. Find the nth and (n+1)th character in the CRDT
2. Create a position identifier between character n and n + 1 and insert character in the CRDT
3. Send out a remote insert change for that character
Local change: deletion at position n
1. Find the nth character in the CRDT
2. Delete that character in the CRDT
3. Send out a remote delete change for that character

I won’t go into the details of how each of these events are handled. It’s non-trivial and a lot of lines of code, but ultimately rather mechanical and I don’t think there’s a whole lot of insights to be had.

That being said, there’s 3 ways in which you could implement this, from easiest (and least performant) to hardest (and most performant).

Array of <position identifier, character>

The simplest thing you can do is to store all your characters as a linear array. When you need to search for a position identifier, use binary search. Searching for newline characters may require linear search. This is only going to scale up to a few hundred characters, since you have worst-case quadratic complexity when making large changes. However, it is perfectly reasonable to start with this as a prototype.

Array of array of <position identifier, character>

You can also split the array of characters for the whole document and have one array for each line. This should allow you to support text documents up to a few hundred lines, assuming most text documents have at most a few hundred characters per line. If you only want to build a collaborative text editor as a side project, this approach is probably fine.

Binary search tree of <position identifier, character>

The most efficient approach is to store the characters in a binary search tree. You’ll probably have to supply a custom comparator. Finding, inserting and deleting characters are all O(log(n)) operations as long as you choose an implementation that’s balanced.

You will, however, also need to augment the binary tree with order statistics. That is, you also want to be able to query for the nth character in the tree, by storing the size of the tree at each node. You might also want to be able to query for the nth newline character.

The downside, in addition to requiring a custom data structure, is that the binary tree will require additional storage per character (at the very least a left and right pointer, integer for node size) which is big given how memory hungry this approach is already.

Trie of positions

You can reuse individual positions (“digits”) in a position identifier by storing the whole thing as a trie. Of course, this is a bit more challenging than a regular trie, since you will have to deal with the site tiebreakers and removal of characters. See also TreeDoc.

(Note that in any case, can’t use a hashtable on the client since hashtables don’t provide an ordering to the keys).

Conclusion

I’m not going to pretend writing a real-time collaborative text editor is an easy project. It took me 3-4 days just to write the core of it, and an order magnitude more to write tests, make the UI nice, and add other features on top of it (e.g. seeing the other people’s cursors). It would take even more to make it resilient to failures, implement access control, etc. However, as I hope this article convinced you, there are approaches that can be fairly intuitive and the solution can be understood as a series of layers, and broken down into approachable tasks.

Thanks for reading, and let me know if you have any feedback!

Thanks to Ryan Kaplan for reading over and reviewing this blog post.