Frequently Asked Questions On My Writing Process

Every month of so, someone asks me about my writing process: How did I get started writing? Why do I write? Who am I writing for? How do I approach writing?

I’ve hesitated to write my responses to these because, honestly, who cares about my writing process? But after repeatedly answering similar questions, I decided to put my thoughts down once and for all (so I can stop repeating myself). If you’re thinking about starting to write online but feel unsure how to begin, this is for you.

How did you get started writing?

Years ago, when I was a junior data scientist, I reached out to several more experienced folks—senior data scientists, heads of data science, even CTOs—and asked them: “What makes an effective data scientist?” Was it PhD-level research skills, coding expertise, the ability to analyze and prepare terabyte-level datasets, deep domain knowledge, or something else?

Their responses surprised me. While they acknowledged that these technical skills were important, a majority emphasized something else entirely—communication.

Their best data scientists stood out because they could listen carefully, distill the challenges that business and product faced, identify where machine learning can help, and translate it into requirements and designs for science and engineering. They discussed statistics and machine learning without relying on jargon like “AUC-ROC” or “gradient boosted decision trees” and instead focused on outcomes like “increasing conversion” or “reducing delivery delays”. As a result, these communicators could pinpoint the real needs, making it easier to get buy-in, execute effectively, and earn trust. To encourage me to focus on communication, one mentor even said that it was critical yet easily transferable to any role and industry.

I’ll admit—I was skeptical at first. I had a hard time believing that a non-technical skill like communication would have a big impact on success in a technical role like data science. However, after hearing the same message from a handful of mentors, I decided to test it myself and commit to improving my communication skills for a year. During that year, I volunteered to speak at every internal workshop and external conference, wrote and edited company-wide newsletters, and started this website.

Over the years, I’ve benefitted greatly from this habit. Writing has helped me reinforce my learning and sharpen my thinking, make friends on the internet, and accelerate my career growth. That’s why I continue to practice writing online.

Why do you write?

First, I write to learn. Often, this happens when I’m exploring a topic but can’t find useful resources online. Maybe information is fragmented across papers and tech blogs, or perhaps it lacks a clear overarching framework. Thus, I dive into the literature, run experiments, and simplify what I learn into clear, reusable patterns. Along the way, writing forces me to fill the gaps in my understanding and clarify my thoughts. Examples of this include: RecSys System Design, Feature Store Hierarchy of Needs, LLM Patterns, and Lessons from Applying LLMs.

I also write to share knowledge. Sometimes it’s because I receive the same questions multiple times, so writing it down somewhere makes it easier for me to just share a link. Other times, it’s because I think the information is valuable and can help others. Examples of such writing are OMSCS FAQ, Writing Why What How, Prompting Fundamentals, How to Interview ML/AI engineers, and MacBook Pro Setup.

Occasionally, I write to express disagreement. This can involve pushing back against the anti-pattern of data scientist throwing trained models over the wall to engineers to productionize, technical folks deliberately overcomplicating their work for publication or promotions, or sharing my frustration that academic LLM evaluations don’t translate to real-world product outcomes. Writing these is somewhat cathartic. Some are even cited and debated over, which makes me hopeful that they’ve positively influenced the field. Examples include: Data Scientists Should Be More End-to-End, Start without Machine Learning, Simplicity > Complexity, and LLM Evals that Don’t Work.

Finally, all my writing (and social media posts) serve as my bat signal. It’s my way of saying, “Hey, here’s what I’m thinking and working on! If you’re exploring similar ideas or facing similar challenges, please reach out!” This has been more effective than I’d imagine, leading to enlightening discussions with senpais and fellow practitioners on topics such as RecSys, LLM-powered systems, and evals. I’ve learned so much and made many friends this way.

Who do you write for?

I primarily write for myself. Writing helps me reinforce what I’ve learned and clarify my thoughts. And because I mainly write for myself, I only write about topics I’m actually interested in. The flip side is that I can’t force myself to write on subjects I’m not passionate about, even if people offer to pay me a lot of money.

The second audience is my team. With them I share industry-proven methods, best practices, and design patterns to get better at what we do. Since they are familiar with the field, I can use jargon to keep the writing concise; otherwise, each piece would be excessively lengthy if I had to explain standard concepts and start from the beginning. (My reference point for technical writing is Lilian Weng and Chip Huyen.) As a result, readers sometimes comment on the use of jargon (below), but that’s okay—they might simply not be the intended audience.

I started listening to this article (using a text to speech model) after waking up.

I thought it was very heavy on jargon. Like, it was written to make the author appear very intelligent without necessarily effectively conveying information to the audience. This is something that I’ve often seen authors do in academic papers, and my one published research paper (not first author) is no exception.

I’m by no means an expert in the field of ML, so perhaps I am just not the intended audience. I’m curious if other people here felt the same way.

Hopefully this observation / opinion isn’t too negative. — Source

Third, I write for the leadership at my organization. For example, in early 2023, I received questions about finetuning, RAG, evaluations, etc. Thus, I spent several weeks researching and distilling my thoughts into practical patterns. The result was Patterns for Building LLM-based Systems & Products. Surprisingly, despite how lengthy it was (60+ minutes of reading time), some of them read it. This has allowed our discussions to evolve beyond foundational questions towards more nuanced challenges in the trenches.

There’s a slight tension between balancing the needs of the second and third groups. The team wants details on the data, methodology, ablation studies, etc.—the “how”. On the other hand, leadership is more interested in the bigger picture and what it means for customers, the business, and the organization—the “why”. I think that striving to write for both audiences simultaneously makes me a more effective and practical writer.

Finally, I write for the community. My goal is to help others deepen their understanding and fill knowledge gaps. I’ve gained a lot from other writers on the internet and this is my way of contributing back, by patching gaps of information and knowledge via my writing.

How do you decide what to write about?

I typically write about what’s relevant to my work at that time. For example, in 2020, I started a new role focused on recommendation systems and wanted to consolidate my learning from Lazada, Alibaba, and Amazon. And from 2023 onward, my writing has shifted towards topics related to LLMs as I began experimenting and building extensively with them.

How did you find your niche?

I’ve never thought about my niche. I simply wanted to practice and improve my writing. That said, at the end of 2020, I noticed that my machine learning and data science write-ups had the highest median open rates (below), suggesting that they were interesting to my audience.

Email open-rate in 2020 by themes

With this insight, I started writing teardowns, of machine learning and recommendation systems in 2021 which became popular with the community. I considered popularity as a proxy for usefulness and thus continued writing similar pieces, including surveys.

How did you choose what platform to write on?

I started with WordPress because it was the easiest platform back then. I could just start writing and not have to concern myself with building the site from scratch or hosting it.

After a few years, when I wanted more customization than WordPress allowed, I switched to Jekyll which was, and still is, free on GitHub Pages. This also allowed me to tinker with the frontend via basic CSS and JavaScript.

What’s your writing pipeline? Do you have a template?

I usually start with a bullet-point outline in Obsidian. At the top, I jot down brief notes on what I’m sharing, why I think it’s valuable, and who the intended audience is. Then, I draft section headers and gradually add bullet points while reviewing literature or when something comes to mind. Drafting with bullet points helps me stay flexible—I can rearrange, remove, or expand points without worrying about the overall structure. I also think of bullet points as less precious than fully formed sentences and paragraphs, which makes writing drafts more carefree. The introduction and conclusion stay empty at this stage.

Once the bullet-point outline is fleshed out, I convert it into prose. By this stage, the bullet points usually have enough detail to make writing sentences straightforward. Although LLMs can do this for you now, I enjoy crafting the sentences and structuring the paragraphs myself. After finishing the main body, I add the introduction and conclusion, spending extra effort on an introduction that tries to hook the reader without overselling the content.

Toward the end, I do the standard spelling and grammar checks. Finally, I read through it one last time and edit for clarity and readability. The polished markdown can then be directly pasted into my Jekyll site and I add images where necessary.

I don’t have a specific template.

Can I write about a topic I just started learning about?

Yes! Think of expertise as a ladder—wherever you are on the ladder, there’s going to be people above and below you. You want to learn from the people above you on the ladder. Similarly, people below you would like to learn from you. Thus, even if you’ve just started learning about something, write about what you’ve learned, because other beginners can learn from you. As an added benefit, writing about it helps reinforce your understanding and learning of that topic. Related to this, my friend Swyx has an inspiring essay on learning in public.

What’s the right frequency to write?

The ideal frequency is just slightly beyond your comfort zone yet still sustainable.

If you’re just starting to write online, I recommend focusing on quantity over quality, at least while you build the habit. Aim to publish weekly, or at the very least, monthly. Once writing consistently becomes second nature, you can gradually shift your focus to improving quality.

How to overcome the perfectionist mindset?

It helps to timebox each piece you write. Set a deadline by which you’ll publish and stick to it. There’s always something that can be improved on, so accepting that writing is inherently imperfect helps ease the pressure. Also, publishing a piece doesn’t prevent future updates or improvements—the important part is hitting “publish”.

How did you build a brand for yourself?

I don’t think I have a brand. Even if I do, I didn’t build it consciously. But if there’s anything, I think it’s the consistency of my output, and my writing being at the practical intersection of recommender systems, LLMs, and production.

How do you balance writing (so much) and your day job?

When working on a substantial piece, I spend an hour or two each night reading and taking notes on research papers and technical write-ups. Over the weekday nights, this adds up to around 5 – 10 papers. On weekends, aside from snowboarding in winter and hiking in summer, I can spend up to eight hours a day researching and writing.

Also, I’ve an incredibly understanding wife ❤️

How do you set boundaries between work vs. personal writing?

Given my role in big tech, I intentionally avoid writing about my day-to-day work, despite how meaningful and useful the sharing can be. To get around this, I focus my writing on the bigger picture and design patterns. If I want to write about the details, I draw references from publicly available sources like papers and technical articles. As a result, I avoid mentioning my employer in my writing, except through publicly available references.

Other relevant write-ups on writing

If you found this useful, please cite this write-up as:

Yan, Ziyou. (Mar 2025). Frequently Asked Questions On My Writing Process. eugeneyan.com.
https://eugeneyan.com/writing/faq/.

or

@article{yan2025writingfaq,
  title   = {Frequently Asked Questions On My Writing Process},
  author  = {Yan, Ziyou},
  journal = {eugeneyan.com},
  year    = {2025},
  month   = {Mar},
  url     = {https://eugeneyan.com/writing/faq/}
}

Share on: