tl;dr - I co-authored an O’Reilly book with my colleagues Lak and Mike! You can order it here and 100% of the royalties go to Girls Who Code. This post is about the writing process.
Deciding to write
Back in January of this year, Lak graciously reached out to me asking if I was interested in co-writing an O’Reilly book on machine learning. As I read the email, my reaction was almost an immediate no because:
- It sounded like something that would take a lot of time.
- With a quickly evolving field like ML, wouldn’t a book be obsolete before it hit the shelves?
I let Lak’s request sit in my inbox for a few days even though I was sure I had made up my mind, because saying no to projects is much harder than saying yes. A few days later on a long car ride with my fiancé, I mentioned the book offer in passing.
“You have an opportunity to write a book and you’re saying no?” he said, nearly pulling off the road.
I suppose I had considered this offer as more of a burden, but from his perspective it was an opportunity. Ok, he wouldn’t be the one writing it, but he has written an O’Reilly book before!
It would no doubt be hard, but it also provided a chance to learn a lot along the way and have a book at the end of it. With his advice in mind, I decided to learn more about the book before saying no. The plan for the book was to catalog a series of patterns, each of which would outline a common challenge in ML and provide some approaches for solving it. I was intrigued, but some doubts started to creep in as I read through the pattern ideas:
-
I was very familiar with some, but much less so with others. Didn’t I need to have all this knowledge in my head before writing for this to work?
-
Why me? Why was I qualified to do this? Objectively, I have created a lot of ML content over the years to educate app developers, data scientists, and customers. I’d like to think I’m a decent writer, but a book? That seemed like a different beast, and I wasn’t sure I was the best person for the job.
There can always be a million reasons not to do something, but I ultimately decided to ignore them all and give it a try.
Starting with an idea
This happened before I joined the project, so I asked Lak what he did before reaching out to O’Reilly with the idea for the book. Publishers get cold proposals all the time, so it helps if you can clearly communicate what the book will be about and show that you know what you will be writing about.
In our case, Lak wrote a couple of blog posts on machine learning design patterns (Transform and Checkpoints) to gauge whether readers were interested in this idea. They were, but he realized that readers’ interest was very much in how to implement, adapt, and extend these patterns. He also did a talk on the topic at a couple of conferences and discovered that conference attendees were a bit different – they needed a clear explanation of the reason to use the pattern and a prescriptive approach to addressing the problem. In other words, readers who knew the pattern beforehand wanted a technical manual, while conference attendees (who were being introduced to the pattern for the first time) wanted to know why the pattern matters. This helped him write the next set of blog posts, where he started with a crisp problem statement, a canonical solution, and a couple of variations.
Once he had a good idea of how to communicate ML design patterns, Lak reached out to an O’Reilly editor with a short message:
I have a list of about 20 machine learning design patterns that I want to write about. Do you think they would do well as a book? If so, I can see about finding a co-author who would be interested in expanding on these…
The editor replied that since O’Reilly has a solid franchise in design patterns, he was very interested. He was assigned an acquisitions editor who would work with us to craft the actual proposal. And that’s when Lak reached out to me to see if I was interested in co-authoring.
Building an outline
I knew nothing about writing a book going into this, and my first question was: how will we split up the work? Will we alternate paragraphs, sections, chapters, words? I’m sure this varies for every book with multiple authors. We decided to split work by sections. In our case, each section was a design pattern and each chapter had 3-6 patterns. We started by making an outline, with an initial list of 20 patterns.
Chapters in our book are organized by different phases in a typical machine learning workflow. We first determined the topic for each chapter, and then came up with the patterns we wanted to be included in it. In the outline, each pattern had a 1-2 sentence description. It took a few iterations of adding and removing patterns and moving some between chapters before we had something we were happy with. We finished this process with a total of 30 patterns split across 6 chapters, with 2 additional chapters for an intro and conclusion. This brought us to the end of February.
Submitting a book proposal
The first step in the publishing process for many books is writing a proposal and submitting it to an acquisitions editor. Acquisitions editors are in charge of reviewing proposals, providing feedback, and deciding whether a proposal will move to the next stage in the approval process. They also work on overall content strategy, deciding which topics they want to cover. Sometimes people reach out to them cold, and other times these editors may proactively reach out to prospective authors.
The proposal (which includes the outline, but also comparisons with existing titles, and who the intended audience is) went through a few rounds of feedback (which is where I joined the process), including detailed technical feedback, before getting approved. The acquisitions editor focuses on acquiring new content, so once ours got approved we moved to the next step in the process which was working with a development editor. This person was our main point of contact throughout the writing process – more on that in the next section.
I recently had a great conversation with our acquisitions editor, where I learned a lot more about the publishing industry and how this all works. One thing from this conversation especially stood out: she mentioned she rarely receives a cold book proposal from a woman, and she gets many cold proposals each day. I’m sharing this data (with her permission) in hopes that it encourages more women with book ideas to just go for it. Reach out! You’ve got nothing to lose. If your proposal gets rejected, you’ll likely still get some feedback in the process.
Writing time
I assumed we’d begin at the beginning, but we started by writing Chapter 2. Chapter 1 didn’t have any patterns – it was meant to be an overview of the entire book, explain our goals, and set the stage for the intended audience. Without anything written yet this would be hard to write, so we got right into patterns by starting with Chapter 2. Before sitting down to write this chapter, we took our short Chapter 2 outline and worked together to expand it. This is something we did before every chapter, with the goals of:
- Ensuring the three of us had the same understanding of each pattern
- Agreeing on what should and shouldn’t be covered
- Splitting up the writing by pattern
Early on in the outline process, we had decided each pattern should follow the same structure. If you’re reading the book, you’ll notice that every pattern is split into the following:
- Problem: describes the ML challenge a pattern addresses
- Solution: describes one approach to solving the problem, including code snippets and recommended tooling for solving the problem
- Tradeoffs and Alternatives: extended discussion on the pattern, including tools not covered in the Solution section, potential gotchas, and related solutions
Having a structure for each section was extremely useful in ensuring consistency throughout the book.
After outlining Chapter 2 in detail, I had two patterns assigned to me and it was finally time to write. I wish I could tell you this is the part where I sat down at my desk, closed the door, and let the insights flow onto the page. This is how I’d always imagined book writing went. For better or worse (and maybe because the outside world went into chaos while this was happening), my process went more like this:
My writing process:
— Sara Robinson (@SRobTweets) March 29, 2020
* Write 2 sentences
* My glasses are splotchy, where are the lens wipes?
* Edit the last sentence I wrote
* Research the latest keyboards
* Write another 2 sentences
* Refill water, clean the counter
* Stare at my now perfect paragraph
* Open Twitter
It wasn’t a very efficient way to start, but it got the job done and over time my process improved. I also quickly debunked my initial question:
Didn’t I need to have all this knowledge in my head before writing for this to work?
Even if I did have experience implementing a particular pattern, I couldn’t write only what I already knew. I had to do a considerable amount of research before I started writing the bulk of each pattern to find out:
- What writing on this topic already existed?
- Are there any important research papers that cover this?
- What tools are available for solving this pattern?
With all this research I started to have some doubts. If I was just collating information from various sources, was I providing anything useful? Then I thought about some of my favorite non-fiction books and realized most of them work by:
Presenting existing, relevant information + adding a unique angle
I’ve met with a lot of customers and seen production use of ML, and I’ve also built demos to help developers understand how to use different ML tools. Turns out I did have a unique angle! Before writing I did a lot of reading to make sure I fully understood the ins and outs of the pattern. Then I had enough information to write. I still wouldn’t say the words flowed at this point, but I did slowly get my thoughts into an organized format.
I admittedly did most of this initial writing on weekends, having found myself with more time at home than expected due to shelter-in-place. Lak, Mike, and I set rough deadlines for finishing the first draft of each chapter, which also provided a good forcing function for sitting down to write.
Writing code
So far I’ve discussed writing in the context of writing words, but there was also a lot of coding that went into this book. Our book has an associated GitHub repo, and each pattern has at least one Python notebook with example code. In order to write about the patterns, we also had to implement them. For my first few sections I wrote the code as I wrote the sections, but then Lak made the suggestion that it would probably be easier to write the code first and then write the chapter. He was right :) That approach forced me to do all the initial research first, get something working, and from there I had a better foundation to start writing. The process went something like:
- Research & write code
- Write the chapter while referencing the code
- Revise, more research, edit code
Feedback and revisions
Once the three of us finished our sections for a chapter, the next step was reading and reviewing each other’s work. This required a thorough read, adding comments and questions, and implementing feedback. Often, this part of the process involved reworking what we’d written and removing paragraphs to make sections more concise. Then, after the three of us signed off on a chapter, we sent it to ~5 Googlers who provided additional technical review. A few of the same reviewers read every chapter, and others read 1-2 chapters to provide subject matter expertise.
We are extremely grateful to these reviewers, and they are recognized in our Acknowledgements section. They provided feedback from a reader’s perspective and challenged us to think critically about each pattern. Only after we had addressed their feedback did we send the chapter over to O’Reilly for an additional round of review. Our development editor at O’Reilly gave feedback, and once we addressed his comments the chapter was ready for the next phase in the process.
Early release
O’Reilly has an online platform where members can access drafts of in-progress books. Once we had Chapters 2 and 3 completed, it was time to start thinking about preparing the book for early release. In order to do that, we needed a first chapter. It was much easier to write this after we had a better idea of what each chapter would look like. Then, with a Chapter 1 draft complete, our book was ready for early release. Our development editor helped prepare the book, and we got a placeholder cover image of this cute little chick:
If you’ve seen O’Reilly books before, you’ll notice that each one has a distinctive animal on the cover. One of the first questions I asked our editor was how we chose the animal on the cover (I know, I hadn’t even started writing and I was already asking about the cover art. Patience is not my best quality). Interestingly, they keep this process mysterious and the authors don’t choose their cover art. We’d need to wait until we finished a complete first draft before seeing the animal chosen for our cover. You can find out a bit more about this process on the last page of any O’Reilly book.
Burnout
Somewhere in the midst of writing Chapter 5 (out of 8) I hit a wall. For a few months I had spent a good majority of my weekends writing, along with a few weeknights. I was operating something like this:
while nowhere_to_go and extra_time_at_home:
write_book()Yes, the weather was frigid and yes, I couldn’t really leave my apartment because of covid, but humans don’t necessarily operate like computer programs and this approach was no longer working very well. Every time I sat down to write it was taking me longer than before to get words and code out. I really did want to keep writing, but there was some disconnect between this aspiration and actually doing the work. I started by taking a 3 day writing-free weekend where I went on some hikes and tried to figure out how I could still get this done in a timely manner. Over the years I’ve learned that people will very rarely guess how you’re feeling and it’s almost always better to tell them (though very difficult in the moment). In the interest of transparency, I decided to tell my co-authors and my manager that I was burnt out. This went better than expected.
We had been taking things one chapter at a time before, but I proposed a timeline for the rest of the book. It was helpful to see the entire process on paper so I could divide up the remaining time I needed for research and writing. And, because writing this book is very relevant to my job, my manager was ok with me spending a few work days getting it finished. This never would have happened if I hadn’t asked!
For the rest of the book, I made a more conscious effort to separate my writing time from my free time.
Preparing for production
Somehow we made it from writing our first pattern to writing the Acknowledgements section, which was wild. There are so many people to thank, but instead of attempting to list them here, I will share these pictures of our Acknowledgements section:
With a draft of the book fully complete, O’Reilly assigned us a production editor, who was in charge of getting our draft to print. First, the production editor did a full end-to-end review. They were extremely thorough, leaving comments on nearly every page. We reviewed this feedback while simultaneously doing our own end-to-end read through to catch any final changes. Once both the editor’s and our changes were reviewed, O’Reilly sent us the first full PDF of the book. We went through this with a fine-toothed comb looking for any more errors, and then they sent us one last PDF to proof. And with that, it went off to print!
Shipping code, literally
When people started to tell us they were receiving their copies I realized that my code had literally shipped. Seeing other people starting to read the book was extremely rewarding. If you order a copy, I would love to hear what you think! You can send any book feedback to me via Twitter, I’m @SRobTweets. Even if it’s constructive feedback (seriously), please send it my way. And if you find a typo or mistake, let me know about that too. One of the great parts about working with O’Reilly is that the process is not complete when the book goes to print. Anyone can submit errata on our book page here, and these updates are incorporated into future versions.
Print, ebook, and…cake
I’ve gotten really into cake decorating over the last few months, so there was no question that I would attempt a cake version of our book once it was complete. Just like writing a book, this was way more time consuming than I expected but totally worth it in the end. Here’s a picture of me with the finished print version and in-progress cake version:
And here’s how it turned out:
For a video of the cake decorating process, check out my Instagram.
Writing, coding, and cake are some of my favorite things so it was really fun to see them come together in this bizarre way. Thanks for reading!