Things we learned about LLMs in 2024
Significant advancements in large language models (LLMs) have occurred, with GPT-4 being surpassed by models like Google's Gemini 1.5 Pro, which features a 2 million token context and multimodal inputs. Operational costs have decreased, some models costing $0.0375 per token. Emergence of multimodal capabilities allows processing audio and video. Challenges exist, including user navigation issues and inconsistent understanding of functionalities. The article also touches on the environmental impact of increasing infrastructure demands despite lower prompt costs, introducing the concept of "slop" for unwanted AI-generated content.
Deepseek: A new contender in AGI
Deepseek, a Chinese AI startup, is gaining recognition with its R1 model outperforming OpenAI in reasoning benchmarks, backed by hedge fund High-Flyer. Prioritizing foundational and open-source technologies, Deepseek aims to lead a cultural shift in AI innovation in China, moving away from imitation. The company leverages a dynamic team focusing on advanced architectures while eschewing rapid market competition for research and true innovation, with aspirations towards artificial general intelligence.
Salt Typhoon: The telecom hack
Telecom giants like AT&T, Verizon, and Lumen Technologies are responding to major data breaches linked to a Chinese state-sponsored cyberattack named Salt Typhoon, noted as possibly the worst telecom hack in history. Attackers accessed networks, allowing for geolocation tracking and call recording capabilities. The breach led to calls for stricter cybersecurity regulations to protect telecom infrastructures, emphasizing that existing voluntary measures are insufficient against nation-state threats.
Creating DIY apps from Google Sheets
Kevin Purdy describes using Glide, a no-code solution, to create mobile-friendly web applications from Google Sheets, motivated by a desire to aid local restaurants during the pandemic. His experience illustrates the ease of creating functional apps for sorting and accessing data, embracing a "messy middle" approach to small-scale programming. His journey emphasizes convenience and encourages others to tackle personalized tech challenges without complex coding requirements.
The pitfalls of software engineering assumptions
Steven Sinofsky examines common misconceptions in software engineering that lead to complexity, often initiated by phrases like “let’s just.” The article critiques appealing concepts like pluggability and asynchronous processes, warning that they can often result in unforeseen complications. It underscores the necessity of careful architectural planning and a deep understanding of systems, as many seemingly simple solutions can complicate development efforts significantly.