Articles for category: AI Tools

March 28, 2025

ikayaniaamirshahzad@gmail.com

Phi-2 on Intel Meteor Lake

Because of their impressive abilities, large language models (LLMs) require significant computing power, which is seldom available on personal computers. Consequently, we have no choice but to deploy them on powerful bespoke AI servers hosted on-premises or in the cloud. Why local LLM inference is desirable What if we could run state-of-the-art open-source LLMs on a typical personal computer? Wouldn’t we enjoy benefits like: Increased privacy: our data would not be sent to an external API for inference. Lower latency: we would save network round trips. Offline work: we could work without network connectivity (a frequent flyer’s dream!). Lower cost:

March 28, 2025

ikayaniaamirshahzad@gmail.com

Building a Container System from Scratch: Understanding the Magic Behind Docker

Hi builders! I think this has been the shortest time intervals between any of my post (lol). Well, today, I’m excited to share a project I’ve been working on (HNG thingy): building a container system from scratch! Maybe when you hear of “container”, Docker comes right to mind, yeah? Well, if you’ve ever wondered what’s happening under the hood when you run docker run, this post is for you. I’ll demystify containers by creating my own lightweight implementation that captures the core functionality of Docker. 🔍 Introduction: Why Build Your Own Container System? Containers have transformed how we deploy and

March 28, 2025

ikayaniaamirshahzad@gmail.com

how to create large-scale synthetic data for pre-training Large Language Models

In this blog post, we outline the challenges and solutions involved in generating a synthetic dataset with billions of tokens to replicate Phi-1.5, leading to the creation of Cosmopedia. Synthetic data has become a central topic in Machine Learning. It refers to artificially generated data, for instance by large language models (LLMs), to mimic real-world data. Traditionally, creating datasets for supervised fine-tuning and instruction-tuning required the costly and time-consuming process of hiring human annotators. This practice entailed significant resources, limiting the development of such datasets to a few key players in the field. However, the landscape has recently changed. We’ve

March 28, 2025

ikayaniaamirshahzad@gmail.com

5 Next.js Tips Every Developer Should Know 🔥

While working with Next.js, I’ve realized that there are a few repetitive tricks every developer should know when studying and using Next.js. Here are five essential Next.js tricks that every developer should master: 1. Prefetching for Faster Navigation Next.js automatically prefetches links when they appear in the viewport, improving performance. However, you can manually control prefetching for better optimization. import Link from "next/link"; <Link href="/about" prefetch={false}>Go to About</Link> Enter fullscreen mode Exit fullscreen mode This prevents unnecessary prefetching, which is useful for less frequently accessed pages. 2. Middleware for Edge Functions Next.js Middleware runs before the request reaches the server,

March 28, 2025

ikayaniaamirshahzad@gmail.com

Advancing Large Model Training on Consumer-grade Hardware

The integration of GaLore into the training of large language models (LLMs) marks a significant advancement in the field of deep learning, particularly in terms of memory efficiency and the democratization of AI research. By allowing for the training of billion-parameter models on consumer-grade hardware, reducing memory footprint in optimizer states, and leveraging advanced projection matrix techniques, GaLore opens new horizons for researchers and practitioners with limited access to high-end computational resources. Scaling LLMs with Consumer-Grade Hardware The capability of GaLore to facilitate the training of models with up to 7 billion parameters, such as those based on the Llama

March 28, 2025

ikayaniaamirshahzad@gmail.com

Add Authentication to your React App with Frontegg

This is a quick start guide with an accompanying sample app on integrating authentication into a client-side React app using Frontegg’s React SDK. Is this for you? You’re looking to for an IdP, authentication, authorization, OAuth/OIDC login and SSO, user management You’re working with a client-side React app (SPA) The Frontegg React SDK is specifically designed for client-side React apps If that’s not you, you may want to see if any of the other SDKs offered here meet your needs: Most of what will be talked about here is abstract enough to apply to the other tech stacks as well

March 28, 2025

ikayaniaamirshahzad@gmail.com

Speech acts in the Dutch COVID-19 Press Conferences

This section addresses the research questions. We start with assessing the quality of the speech act annotation. Then we describe the press conferences in terms of the speech acts, following the subquestions. And finally we evaluate how well a machine learned speech act classifier can help in reducing the annotation time and costs. 5.1 Identifying speech acts in the Dutch COVID-19 press conferences Section 4.2.2 described that the inter-rater reliability as measured by Krippendorff’s \(\alpha \) was .71 for the single labelled sentences and .70 for all sentences. This is generally considered a viable score. There are some points of attention,

March 28, 2025

ikayaniaamirshahzad@gmail.com

Introducing the Chatbot Guardrails Arena

With the recent advancements in augmented LLM capabilities, deployment of enterprise AI assistants (such as chatbots and agents) with access to internal databases is likely to increase; this trend could help with many tasks, from internal document summarization to personalized customer and employee support. However, data privacy of said databases can be a serious concern (see 1, 2 and 3) when deploying these models in production. So far, guardrails have emerged as the widely accepted technique to ensure the quality, security, and privacy of AI chatbots, but anecdotal evidence suggests that even the best guardrails can be circumvented with relative