benchmark
61 stories and discussions about benchmark, aggregated from every source we track.
Benchmark for tracking model capability after release. - ninjahawk/livenerf
A benchmark where frontier language models drive a real comma-equipped Toyota through a cone course, one command at a time, with a human supervisor ready to brake.
52 Jev-class systems tested on 534 decisions. Jev 1.13.0 leads JevBench v1.3.0 with 74.4; compare open-source, self-hostable and hosted options.
OpenAI's involvement in funding FrontierMath, a leading AI math benchmark, only came to light when the company announced its record-breaking performance on the test. Now, the benchmark's developer Epoch AI…
Uncensored AI models or those fine-tuned for cybersecurity tasks. - JoasASantos/Offensive-Security-AI-Models
This is a submission for the Kaggle Benchmarking Challenge What I Benchmarked Current...
This article is a submission for the Kaggle Benchmarking Challenge. "I told an AI something was...
LLM Benchmark - One prompt, Multiple models, Multiple dates
There's a test suite for security scanners where 47% of the test cases are deliberately designed to...
Discover how Gubernator revolutionizes container orchestration by natively baking in ENS RD 311/2022, EU NIS 2, EU DORA (Reg. 2022/2554), CIS Docker Benchmark, ISO 27001, SHA-256 audit ledger, Cosign, and SBOM into a single sovereign Go binary.
I published a memory benchmark six days ago. I spent most of the build trying to make it fair rather...
This is a submission for the Kaggle Benchmarking Challenge What I Benchmarked ...
Benchmark TypeScript 7 safely, compare real project results, and find compatibility problems before changing your main branch.
On GitHub Actions' standard macOS runner, a Firefox for iOS build takes over 12 minutes and on Avrea with a warm cache it takes just under 2 minutes. Mastodon, built with Tuist, goes from 4 minutes 30 seconds to 12…
C++ vs Node.js HTTP benchmark with wrk: Crow 5.4k req/s, Node http 12k, uWebSockets 57k. Same route, same laptop, with code and why Crow lost to Node.
The World's First Information-Theoretic Search Engine. Deploy a complete, sovereign AI search stack into your VPC, on-prem or on edge.
DoGBench evaluates whether agents identify when user-facing software documentation needs updating and produce useful patches, using rubrics calibrated with project maintainers.
Benchmark of TypeSafe's Jev against Sonnet 5, GPT-5 nano and local LLMs on 770 Reddit AITA verdicts: Brier scores, latency and cost - dchristopoulos/jev-aita
Benchmark for tracking model capability after release. - ninjahawk/livenerf
A browser-first AI poker benchmark. Seat Jev and OpenAI-compatible models at the same no-limit Texas Hold'em table, watch every card and decision as a spectator, and let the tournament run auto...
TypeSafe's Jev against four shipped prompt-injection detectors on 7,803 labeled messages and 296 conversations. It wins the curated benchmark and the newest attack set, loses two older ones, and its probabilities track…
WebAssembly runtimes keep improving every day and quite a few things havechanged since the last time I benchmarked WebAssembly runtimes using libsodium.
Our benchmark shows OpenAI’s Dots defaults to its own platform, powered by Cloudflare, rather than separate third-party services when building apps.
KernelBench: Can LLMs Write GPU Kernels? - Benchmark + Toolkit with Torch -> CUDA (+ more DSLs) - ScalingIntelligence/KernelBench
Exploring products at the intersection of Design & AI
Measurement harness for proxy providers, browser engines and scraping targets. Every claim carries the run it came from. - nodemaven/proxy-benchmark
See how Jev performs in a benchmark of 19 reranker configurations across five datasets, covering accuracy, latency, and prompt-driven relevance for LanceDB search.
Benchmark for tracking model capability after release. - ninjahawk/livenerf
This is a submission for the Kaggle Benchmarking Challenge. Every day, agents book invoices, check...
Introducing webgpu-bench, the micro-benchmark suite behind Web3D Survey’s GPU benchmark, with measurements of shader operations and code patterns available through a webpage and CLI.
Browser stacking work-cell for embodied agents. VLA track: camera and language. Scores written by the harness, not the client.
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
We benchmarked 11 speech-to-text engines on 265 real recordings with a competing voice in the room. See where voice isolation helps.
Choose the best model for the work you do
Our own benchmark of typed-decision models: Jev, its open rebuilds and instruction models on the JevBench Score — Intelligence, Calibration, Speed and Cost, 25 % each, geometric mean.
Measurement harness for proxy providers, browser engines and scraping targets. Every claim carries the run it came from. - nodemaven/proxy-benchmark
Does the code do what it says? A benchmark for catching the gap between documentation and reality. - 02zerocool/truth-benchmark
A single-image handedness decision: probabilities, visual evidence, original prompt, and JSON output.
ScholarCatalyst: a benchmark for retrieving papers that inspire new research.
Benchmark-driven Rust PDF reader, CLI and desktop viewport with reproducible per-workload performance comparisons - marksmayo/blazepdf
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
An 82% on the leaderboard could be about 57% on real finance workflows. Here's what to take off, and why.
The Hutter Prize requires no validation data, so its scores cannot be inflated by data leaks. Three entries, each running on a single CPU core, just improved its record by nearly 10%.
Current AI safety evaluation and benchmarking frameworks predominantly rely on Western-centric culture-agnostic defaults that mask critical regional laws, socio-linguistic nuances, and cultural taboos, leaving…
runescape-bench evaluates AI coding agents on their ability to play RuneScape.
Most benchmarks hand the model a question. EnigmaForge hands it a stack of old documents and no question at all. Buried in the letters, receipts, and logbook margins is a small logic puzzle whose solution is unique -…
See which LLMs lead the July 2026 EDA Benchmark for exploratory data analysis, comparing accuracy, repeatability, and reliability-adjusted scores.
FAB: a benchmark for AI agents doing financial due diligence - SecondState-ai/finance-agents-benchmark
How UIUC++ SRSE students turned real-world failure postmortems into SREGym benchmark problems, with lessons on fault fidelity, mitigation oracles, and quality control.
Frontier language models are rarely used in clinical workflows because the realistic, longitudinal benchmarks needed to develop them are scarce. Real electronic health record (EHR) data cannot be openly shared due to…
A benchmark for AI-discovered numerical solvers: fast, accurate, and general solutions to large sparse linear systems.
A free, public dataset for measuring name and entity accuracy in speech recognition. See the audio, the annotations, the scorer, and how Modulate scores.
Five free tools for B2B outbound: a LinkedIn message generator, reply rate calculator, ICP fit scorer, voice note script generator and ROI calculator. All built from 389,890 measured prospects. No sign-up.
Compete in short reflex, memory, focus, and precision tests. Save your scores, compare rankings, and climb HumanRank leaderboards.
Claude Opus 5.5 nerf claims finally meet a day-0 baseline: livenerf tests it daily for 30 days. First possible call: around October 24.