Imagine you’re trying to evaluate multiple AI agents on a battery of tests, but you can’t observe all agent-item pairs due to computational constraints. How do you infer the missing performance data? We’ll review total variation distance (TVD) and mutual information (MI) and how it can be connected with the Rasch model in item response theory (IRT) using the principle of maximum entropy. This sparse evaluation problem led me to use information theory through maximum entropy to make a precise solution that is naturally compatible with how TVD-MI works. We then validate assumptions and show how it can reduce the sampling requirements to estimate TVD-MI.

Taking a measurement theory approach to examine one human rule and three GPT-o3 prompts

Something fundamental is happening during attention training that we’ve been missing. While studying the gradient dynamics of self-attention layers, I discovered a conservation law that puts strict constraints on how attention mechanisms can evolve during learning.

When I first conceived the Branching Directive, I imagined tending a garden of knowledge, guided by Voltaire’s famous advice to “cultivate our own garden.” What I discovered instead was a tree – one that grows according to attachment patterns found throughout nature. This perspective transforms my understanding of how research and knowledge naturally organize themselves.

In the realm of artificial intelligence and knowledge management, we often find ourselves grappling with fundamental questions about the nature of understanding and the structure of knowledge. As Voltaire wisely counseled, “We must cultivate our own garden.” This philosophy has been a guiding principle in the development of the Branching Directive, a novel system that seeks to bridge the gap between human cognition and artificial intelligence.

I saw an interesting website the other day where the winner always had to pay one dollar more than the previous visitor. This bought the winner the title “King of the Internet”. I was a bit surprised to see the price hang at around $40 dollars for almost two weeks as of the time of writing. In comparison the first several bids were all within the span of a few days. This got me thinking about why that might be the case. Hence the question of this post: What would happen if the sequence of visitors to the website had identically distributed buy-out tendencies \(\lbrace w_i \rbrace\)?

It’s fair to say that we spend a lot of our lives floating from task to task. Some are work and some are not. Nevertheless, they are tasks. Things that we want to do. Some are complicated are require planning. However, since planning is such an integral step in completing more complicated tasks it seems as though we might want to “meta-plan” or plan our planning. I became fascinated by the concept after I realized it should be possible to frame the whole thing as a sort of optimization problem. In practice, the problem remains complicated by the fact that optimization occurs in a necessarily online fashion. However, any reasonable method of planning is especially well behaved in the formalism I’ll present. Moreover, even when the method is faulty due to human bias, as long as it is consistent we still end up being to prove that meta-planning, in a certain sense, is rational.

Can the Quran be analyzed algorithmically? Many would say no. The Quran is a large and sophisticated document that does not yield to the reductionist approaches traditionally found in statistical sciences. However, a reduction of scope helps in reaching an affirmative response to this question. Here, the Quran is discussed in its chronological order and consideration is given to how the themes of the later Medinan surahs relate to the earlier Meccan surahs. Using natural language processing methods and machine learning a succinct overview of how the themes of Medinan surahs contrast with the earlier Meccan surahs of the Quran is provided. Although algorithmic analysis isn’t able to do close reading of the text, this analysis complements and focuses standard analysis of Medinan surahs so that themes can be compared that might otherwise be missed using more traditional approaches.

Robot overloads ruling humanity has been a common theme in movies and dystopian novels for the past half century. Today, smaller-scale, more realistic concerns, like whether Alexa is eavesdropping on conversations, has captured the collective conscience. As technology rockets forward, the question of what happens once artificial intelligence (AI) surpasses human intelligence is becoming a very real concern. The gap between fiction and reality is shrinking faster than ever and although claims of impending robot overlords are premature, it is not clear whether the ever-quickening pace of AI development is one that puts the world on a path towards utopia or dystopia. Emerging AI applications are radical in themselves and worthy of concern. The potential for unintended consequences, misaligned application, and loss of control are all significant risks inherent in the technology. Yet, currently, funding for research into AI safety is less than 2 percent of the funding into AI development. This is at a time when entities such as China and Facebook are increasingly looking towards using AI for social management while new applications such as driverless cars loom on the horizon. The current trajectory of AI development is reckless, misdirected, and ultimately poses a significant threat to the potential benefits of the technology.