The new company will develop AI safety and capabilities in tandem.

Formation of Safe Superintelligence, Inc. (SSI)

Key Founders and Background:
- Ilya Sutskever: Co-founder and former chief scientist of OpenAI.
- Daniel Levy: Former OpenAI engineer.
- Daniel Gross: Investor and former partner at Y Combinator.

These prominent figures have joined forces to create Safe Superintelligence, Inc. (SSI), a new company aimed at advancing AI safety and capabilities. The company is based in the United States, with offices in Palo Alto and Tel Aviv.

Objectives and Mission

SSI’s Mission:
- Focus on AI Safety: Ensuring the development of artificial intelligence with an emphasis on safety and ethical considerations.
- Balancing Capabilities and Safety: Developing AI technologies where safety measures and advancements in capabilities go hand-in-hand.
- Insulation from Commercial Pressures: Operating with a business model that shields from short-term commercial pressures, allowing a singular focus on safety and long-term progress.

Founders’ History and Motivations

- Ilya Sutskever: Left OpenAI on May 14, following a turbulent period involving the firing and subsequent rehiring of CEO Sam Altman. Sutskever's role at OpenAI had been ambiguous after stepping down from the board.
- Daniel Levy: Departed OpenAI shortly after Sutskever, following concerns about the direction of AI safety within the organization.
- Previous Involvements: Sutskever, along with Jan Leike, had been leading OpenAI’s Superalignment team focused on steering and controlling advanced AI systems, referred to as artificial general intelligence (AGI).

Industry Context and Reactions

Concerns from Other Tech Leaders:
- Vitalik Buterin: Ethereum co-founder highlighted the risks associated with AGI, emphasizing that while it poses significant risks, other threats such as corporate megalomania and militaries are also substantial.
- Elon Musk and Steve Wozniak: Among over 2,600 tech leaders and researchers who advocated for a six-month pause in AI system training to consider the profound risks posed by advanced AI technologies.

Company Hiring Efforts:
- SSI is actively recruiting engineers and researchers to further its mission of creating safe and advanced AI systems.

Strategic Implications

Potential Impact:
- AI Safety Innovations: SSI’s focused mission could lead to significant advancements in AI safety protocols and technologies, influencing the broader AI industry.
- Industry Influence: Given the founders’ backgrounds and reputations, SSI may set new standards for balancing AI capabilities with safety, potentially impacting regulatory and industry practices.
- Long-term Vision: By insulating from short-term commercial pressures, SSI aims to pioneer long-term solutions for safe AI development, potentially mitigating the risks associated with rapid AI advancements.

Conclusion

SSI represents a critical development in the AI industry, bringing together leading minds to address the safety and ethical considerations of AI advancements. The company’s focus on balancing safety with capabilities, and its insulation from commercial pressures, positions it uniquely to influence the future trajectory of AI development.