Etched is making significant strides in the AI hardware industry with its innovative AI accelerator chip. Founded in 2022 by Harvard dropouts Gavin Uberti and Chris Zhu, the Silicon Valley startup has developed a custom application-specific integrated circuit (ASIC) called Sohu, designed specifically for transformer models, the backbone of today's advanced AI systems.

Etched claims that its Sohu chip can handle AI workloads up to 20 times faster than Nvidia's leading GPUs while consuming much less power. With $120 million in new funding and partnerships with major cloud providers, Etched is positioning itself as a strong competitor to Nvidia in the AI chip market.

The recent funding round was led by Primary Venture Partners and Positive Sum Ventures, with notable investors such as Peter Thiel, Github CEO Thomas Dohmke, and former Coinbase CTO Balaji Srinivasan participating. As transformer models continue to drive AI advancements, Etched's specialized hardware has the potential to reshape AI computing.

Etched's approach focuses on the complexities of GPUs and TPUs, particularly the need for handling arbitrary CUDA and PyTorch code, which requires sophisticated compilers. While other AI chip developers like AMD, Intel, and AWS have invested heavily in software development with mixed results, Etched is concentrating on running transformers exclusively. This allows for streamlined software development tailored to these models.

Most AI companies use transformer-specific inference libraries like TensorRT-LLM, vLLM, or HuggingFace's TGI. Although these frameworks can be inflexible, they generally meet the needs of different applications—text, image, or video—because transformer models share fundamental similarities. Users can adjust model hyperparameters without changing the core model code. However, prominent AI labs often require custom solutions, employing engineers to optimize GPU kernels meticulously.

Etched aims to eliminate reverse engineering by making its entire software stack open source, from drivers to kernels. This openness allows engineers to implement custom transformer layers as needed, enhancing flexibility and innovation.

Etched's approach to AI hardware is similar to the advancements seen with Groq's LPU Inference Engine. Groq's LPU, a dedicated Language Processing Unit, has set new benchmarks in processing efficiency for large language models, surpassing traditional GPUs in specific tasks. According to ArtificialAnalysis.ai, Groq's LPU achieved a throughput of 241 tokens per second with Meta AI's Llama 2-70b model, demonstrating its capability to process large volumes of simpler data more efficiently than other solutions.

This performance highlights the potential for specialized AI hardware to revolutionize the field by offering faster and more efficient processing tailored to specific AI workloads. Etched claims its ASIC achieves processing rates of up to 500,000 tokens per second, surpassing Groq's performance.

The introduction of ASICs for Bitcoin mining in 2013 marked a significant shift, greatly improving mining efficiency compared to CPUs and GPUs. This change led to a rapid centralization of mining power and the rise of industrial-scale mining operations. Similarly, Etched's ASICs could bring a revolutionary change to AI, offering unprecedented computational power and energy efficiency.

Etched's journey began in 2022 when AI technologies like ChatGPT were not yet widespread, and image and video generation models mainly used U-Nets and CNNs. Since then, transformers have become the dominant architecture across various AI domains, validating Etched's strategic focus.

The company is moving quickly towards one of the fastest chip launches in history, attracting top talent from major AI chip projects, partnering with TSMC for their advanced 4nm process, and securing essential resources such as HBM and server supply for initial production. Early customers have already committed tens of millions of dollars to Etched's hardware.

This rapid progress could dramatically accelerate AI capabilities, making models 20 times faster and cheaper overnight. Current limitations, such as slow response times and high costs, could be significantly reduced. Real-time applications, from video generation to AI-driven conversations, could become feasible, addressing current bottlenecks faced even by leading AI firms during peak usage periods.

Etched's advancements promise to make real-time video, calls, agents, and search a reality, fundamentally transforming AI capabilities and their integration into everyday applications.