notacompany
Subscribe

Updated 18h ago

Interesting

A collection of stuff that catches Tiago's eye daily.

Get it by email

One brief each weekday morning. One-click unsubscribe.

3 Sept

Interesting · arxiv.org

The Emergent Symbolic Structure of Artificial Neural Networks

Modern systems in artificial intelligence (AI) somehow excel in domains for which they seem poorly suited. Intelligence has traditionally been modeled as operating over structured combinations of symbols, such as logical formulas. However, the strongest modern AI systems are based on neural networks, which instead represent information in continuous vectors. Vectors seem inadequate for capturing the structure of language, logic, and other cognitive domains, yet neural networks achieve impressive performance in these areas. How do they do it? In this work, we propose a potential answer: Despite appearances, perhaps the internal representations of neural networks implicitly realize symbolic structure. In support of this hypothesis, we show that the vector representations of a variety of neural networks can be closely approximated with symbolic structures: we can replace the network's entire representation-generating process with a closed-form equation instantiating a symbolic structure, and the network's behavior remains largely unchanged. This finding holds for both small-scale neural networks trained to manipulate lists as well as large language models (LLMs) operating in four domains that are central in symbolic traditions: arithmetic, logic, computer code, and language. Further, our symbolic approximation allows us to modify an LLM's behavior in targeted ways via precise interventions on its internal representations, showing that the LLM's behavior is reliant on the symbolic structures we have identified. This work provides a potential way to reconcile longstanding symbolic conceptions of intelligence with the vector-based nature of modern AI.

Neural Networks

Interesting · signalundefied.bearblog.dev

i wanna live an npc life

the general consensus around being an npc is that you’re giving up your autonomy—essentially becoming a side character in someone else's story or watching li...

Npc LifeMinimalism

31 Aug

28 Aug

27 Aug

26 Aug

Interesting · spacex.com

SpaceX Starbase LA

A spaceport on the Louisiana Gulf Coast enabling a multiplanetary future

Star PortStarbaseSpacex

Interesting · arxiv.org

Black hole singularity is a surface not a point

It is widely repeated in the popular literature and elsewhere that the singularity at the center of a black hole is a point. It is not true. Two observers who free-fall into a spherical black hole along two different angular trajectories at the same time $t$ do not encounter each other at the central singularity; rather, they lose causal contact with each other already well away from the singularity. Counterintuitively, in general relativity two points can be spatially close yet causally distant. The singularity is a surface, not a point. The story for rotating black holes is more complicated, but the same conclusion holds. For a rotating black hole, the singular surface almost certainly resides at its inner horizon, where even the tiniest classical or quantum perturbations ignite the exponential mass inflation instability, precipitating collapse to a spacelike singular surface. There are implications for quantum gravity. We argue that, whatever the ultimate theory of quantum gravity may be, the quantum states of a black hole probably reside at its effectively 2-dimensional singular surface, which coevolves unitarily with, and in thermodynamic equilibrium with, the hot atmosphere of trapped Hawking radiation that the black hole generates within its event horizon.

Astrophysics

25 Aug

22 Aug

Interesting · cymerys.com

I'm becoming AI-blind - Rafal Cymerys

Recently I've been catching myself having these little moments at work, when I'm trying to read a document someone has sent me and my brain somehow refuses to analyze it. It feels like I'm reading it, but I'm unable to focus on its content. I sat down to analyze these situations and realized they all have a common denominator: the documents all show a strong trace to AI. My brain learned to quickly spot signs of AI-generated content, at least the low effort one, and it now ignores it and moves on without thinking much about it.

AI saturation

Interesting · hai.stanford.edu

Computational Agents Exhibit Believable Humanlike Behavior | Stanford HAI

Generative agents rely on a large language model to remember their interactions, build relationships, and plan coordinated events, with implications for both gaming and social science.

World ModelsSimulationCool As F

21 Aug

20 Aug

19 Aug

17 Aug

16 Aug

15 Aug

14 Aug

13 Aug

Get this by email

The same feed, once a day, in your inbox. One-click unsubscribe.

Prefer a reader? Subscribe by RSS instead, or browse the past issues.

Where to next

Curious who picks these?

Meet the peopleGet them by email