{"benchmarks":[{"label":"Aegis 2.0","kind":"mixed"},{"label":"OpenAI Mod","kind":"mixed"},{"label":"ToxicChat","kind":"mixed"},{"label":"WildGuardMix","kind":"mixed"},{"label":"CoSApien","kind":"mixed"},{"label":"DynaGuardrail","kind":"mixed"},{"label":"SST","kind":"unsafe-only"},{"label":"HarmBench","kind":"unsafe-only"}],"copy":{"about":"

How to Read This Leaderboard

\n

Every score comes from unsafe-eval, our open-source framework for reproducible AI safety evaluations. For each benchmark, the model classifies each prompt as safe or unsafe. unsafe-eval scores those predictions against the benchmark's labels.

\n

Benchmarks

\n

The leaderboard covers eight benchmarks. Six include both safe and unsafe prompts:

\n\n

Two include only unsafe prompts:

\n\n

Metrics

\n

Each prediction falls into one of four categories. For an unsafe prompt, the model either catches it (true positive, TP) or misses it (false negative, FN). For a safe prompt, the model either lets it through (true negative, TN) or wrongly flags it (false positive, FP). The table offers three metric views:

\n\n

We calculate an average only for models with results on every benchmark in that group.

\n

For hosted models, we also report refusal rate, the share of prompts the provider refused to process. We count each refusal as an unsafe prediction.

\n

Cost, Output Tokens and Latency

\n

Cost is the average price, in US dollars, of processing 1,000 prompts across all benchmarks. It includes prompt tokens a provider writes to or reads from its cache, at the provider's prices for them. We do not report cost for models run on our own hardware.

\n

Output tokens is the number of tokens a model generated across all benchmarks. We report this for local models too. Models that only classify prompts and generate no tokens have no value here.

\n

Latency is the median time to process a prompt across all timed samples in the included benchmarks. For each sample, we sum the recorded model call times. Hosted model latency includes the network round trip.

\n

Numbered notes below the table identify benchmarks omitted from a cost, output token or latency figure.

\n

Model Types

\n\n

See the unsafe-eval repository for benchmark prompts and scoring details.

\n","request":"

To suggest a model, open a discussion in this Space's Community tab with its name, a link, and whether it has open weights or runs behind a hosted API.

\n","citation":"@software{conia2026unsafeeval,\n author = {Conia, Simone and Orlando, Riccardo},\n title = {{Unsafe-Eval}: A framework for reproducible {AI} safety evaluation},\n year = {2026},\n url = {https://github.com/principled-intelligence/unsafe-eval},\n license = {Apache-2.0}\n}"},"rows":[{"Model":"ScopeGuard-2 Pro","Avg balanced acc":0.8827433094350162,"Avg F1":0.8666959640455576,"Avg unsafe recall":0.996875,"Aegis 2.0 BA":0.8077681338918294,"Aegis 2.0 F1":0.8324074074074074,"OpenAI Mod BA":0.8339387427947746,"OpenAI Mod F1":0.878731343283582,"ToxicChat BA":0.9223519587642586,"ToxicChat F1":0.7832335329341317,"WildGuardMix BA":0.8808471222264326,"WildGuardMix F1":0.8685857321652065,"CoSApien BA":0.9460180623973727,"CoSApien F1":0.935672514619883,"DynaGuardrail BA":0.9055358365354291,"DynaGuardrail F1":0.9015452538631347,"SST unsafe recall":1.0,"HarmBench unsafe recall":0.99375,"Refusal rate":0.0,"Cost":0.042118738494331946,"Output tokens":null,"Latency":0.11791809974238276,"Missing":{"Refusal rate":[],"Cost":[],"Output tokens":["Aegis 2.0","OpenAI Mod","ToxicChat","WildGuardMix","CoSApien","DynaGuardrail","SST","HarmBench"],"Latency":[]},"Type":"Task-specific","Local":true,"Open weights":false,"Params (B)":null,"Link":"https://principled-intelligence.com/products/orbitals","Org":"principled-intelligence"},{"Model":"Jev","Avg balanced acc":0.8802730446391084,"Avg F1":0.8756072165800176,"Avg unsafe recall":0.99375,"Aegis 2.0 BA":0.8399711585618689,"Aegis 2.0 F1":0.8587464920486436,"OpenAI Mod BA":0.8396858692315563,"OpenAI Mod F1":0.8849721706864564,"ToxicChat BA":0.8518661657103694,"ToxicChat F1":0.7837837837837839,"WildGuardMix BA":0.9069919863023311,"WildGuardMix F1":0.8966442953020134,"CoSApien BA":0.9367816091954022,"CoSApien F1":0.9277108433734941,"DynaGuardrail BA":0.9063414788331225,"DynaGuardrail F1":0.9017857142857144,"SST unsafe recall":1.0,"HarmBench unsafe recall":0.9875,"Refusal rate":0.0,"Cost":0.027999696153473502,"Output tokens":340593,"Latency":0.36848201300017536,"Missing":{"Refusal rate":[],"Cost":[],"Output tokens":[],"Latency":[]},"Type":"General-purpose","Local":false,"Open weights":false,"Params (B)":null,"Link":"https://docs.typesafe.ai/models","Org":"typesafeAI"},{"Model":"GPT-5.6 Luna `reasoning: default (medium)`","Avg balanced acc":0.8762077329781847,"Avg F1":0.8610439718117862,"Avg unsafe recall":0.985625,"Aegis 2.0 BA":0.8451142235709468,"Aegis 2.0 F1":0.8622472966619651,"OpenAI Mod BA":0.8610770035358186,"OpenAI Mod F1":0.8949858088930936,"ToxicChat BA":0.8262590630135893,"ToxicChat F1":0.7124463519313303,"WildGuardMix BA":0.8902797075210869,"WildGuardMix F1":0.8776486671223513,"CoSApien BA":0.9367816091954022,"CoSApien F1":0.9277108433734941,"DynaGuardrail BA":0.8977347910322637,"DynaGuardrail F1":0.8912248628884826,"SST unsafe recall":0.99,"HarmBench unsafe recall":0.98125,"Refusal rate":0.00019377967251235347,"Cost":0.14802638310241256,"Output tokens":444465,"Latency":1.4822617031168193,"Missing":{"Refusal rate":[],"Cost":[],"Output tokens":[],"Latency":[]},"Type":"General-purpose","Local":false,"Open weights":false,"Params (B)":null,"Link":"https://developers.openai.com/api/docs/models/gpt-5.6-luna","Org":"openai"},{"Model":"Claude Opus 5.5 `reasoning: default (medium)`","Avg balanced acc":0.8759602301247448,"Avg F1":0.8611726912568426,"Avg unsafe recall":0.975,"Aegis 2.0 BA":0.8396701855963866,"Aegis 2.0 F1":0.853868194842407,"OpenAI Mod BA":0.844284707300158,"OpenAI Mod F1":0.8494288681204569,"ToxicChat BA":0.842952862348654,"ToxicChat F1":0.7630402384500746,"WildGuardMix BA":0.89755308548412,"WildGuardMix F1":0.8859709744298548,"CoSApien BA":0.9189244663382594,"CoSApien F1":0.9079754601226993,"DynaGuardrail BA":0.912376073680891,"DynaGuardrail F1":0.9067524115755626,"SST unsafe recall":0.95,"HarmBench unsafe recall":1.0,"Refusal rate":0.04699157058424571,"Cost":5.349124115880244,"Output tokens":456142,"Latency":3.0405499229673296,"Missing":{"Refusal rate":[],"Cost":[],"Output tokens":[],"Latency":[]},"Type":"General-purpose","Local":false,"Open weights":false,"Params (B)":null,"Link":"https://platform.claude.com/docs/en/models/opus-5-5/overview","Org":"anthropic"},{"Model":"Clef `quantization: fp8`","Avg balanced acc":0.8739679129250741,"Avg F1":0.8665521765373416,"Avg unsafe recall":0.9918750000000001,"Aegis 2.0 BA":0.8435774209648241,"Aegis 2.0 F1":0.8633828996282529,"OpenAI Mod BA":0.8163221801562126,"OpenAI Mod F1":0.8659217877094972,"ToxicChat BA":0.8356214970579168,"ToxicChat F1":0.7455621301775148,"WildGuardMix BA":0.8979972773076221,"WildGuardMix F1":0.8865295288652953,"CoSApien BA":0.9367816091954022,"CoSApien F1":0.9277108433734941,"DynaGuardrail BA":0.9135074928684672,"DynaGuardrail F1":0.9102058694699956,"SST unsafe recall":0.99,"HarmBench unsafe recall":0.99375,"Refusal rate":0.0,"Cost":null,"Output tokens":0,"Latency":0.16255088988691568,"Missing":{"Refusal rate":[],"Cost":["Aegis 2.0","OpenAI Mod","ToxicChat","WildGuardMix","CoSApien","DynaGuardrail","SST","HarmBench"],"Output tokens":[],"Latency":[]},"Type":"General-purpose","Local":true,"Open weights":true,"Params (B)":27,"Link":"https://huggingface.co/Cloudflare/clef","Org":"Cloudflare"},{"Model":"GPT-5.6 Sol `reasoning: default (medium)`","Avg balanced acc":0.8726231207287838,"Avg F1":0.8588749999359262,"Avg unsafe recall":0.9853125,"Aegis 2.0 BA":0.8234744838800828,"Aegis 2.0 F1":0.8529014844804319,"OpenAI Mod BA":0.8586354696044658,"OpenAI Mod F1":0.8930936613055819,"ToxicChat BA":0.8236735119357865,"ToxicChat F1":0.7045454545454546,"WildGuardMix BA":0.8910558151937462,"WildGuardMix F1":0.8783877692842251,"CoSApien BA":0.9308292282430213,"CoSApien F1":0.9212121212121211,"DynaGuardrail BA":0.9080702155156003,"DynaGuardrail F1":0.9031095087877421,"SST unsafe recall":0.98,"HarmBench unsafe recall":0.990625,"Refusal rate":0.0004844491812808836,"Cost":2.555017730840035,"Output tokens":324336,"Latency":1.4291088845347986,"Missing":{"Refusal rate":[],"Cost":[],"Output tokens":[],"Latency":[]},"Type":"General-purpose","Local":false,"Open weights":false,"Params (B)":null,"Link":"https://developers.openai.com/api/docs/models/gpt-5.6-sol","Org":"openai"},{"Model":"ScopeGuard-2 4B","Avg balanced acc":0.8717405445968275,"Avg F1":0.8561977763299141,"Avg unsafe recall":0.9871875,"Aegis 2.0 BA":0.8079245748756863,"Aegis 2.0 F1":0.8335638543107423,"OpenAI Mod BA":0.7898603863251361,"OpenAI Mod F1":0.8561085972850679,"ToxicChat BA":0.9135916925240257,"ToxicChat F1":0.7578947368421054,"WildGuardMix BA":0.8739617980997292,"WildGuardMix F1":0.8614232209737827,"CoSApien BA":0.9373973727422003,"CoSApien F1":0.9248554913294796,"DynaGuardrail BA":0.9077074430141869,"DynaGuardrail F1":0.9033407572383073,"SST unsafe recall":0.99,"HarmBench unsafe recall":0.984375,"Refusal rate":0.0,"Cost":null,"Output tokens":null,"Latency":0.03354752901941538,"Missing":{"Refusal rate":[],"Cost":["Aegis 2.0","OpenAI Mod","ToxicChat","WildGuardMix","CoSApien","DynaGuardrail","SST","HarmBench"],"Output tokens":["Aegis 2.0","OpenAI Mod","ToxicChat","WildGuardMix","CoSApien","DynaGuardrail","SST","HarmBench"],"Latency":[]},"Type":"Task-specific","Local":true,"Open weights":true,"Params (B)":4,"Link":"https://huggingface.co/principled-intelligence/scope-guard-v2-4B-q-2609","Org":"principled-intelligence"},{"Model":"GPT-5.6 Terra `reasoning: default (medium)`","Avg balanced acc":0.8716344282936008,"Avg F1":0.8610323766795304,"Avg unsafe recall":0.9984375,"Aegis 2.0 BA":0.8097206689394817,"Aegis 2.0 F1":0.8468626590609917,"OpenAI Mod BA":0.8275719954068466,"OpenAI Mod F1":0.8741845293569431,"ToxicChat BA":0.8741286310275,"ToxicChat F1":0.7552631578947367,"WildGuardMix BA":0.9000785931820414,"WildGuardMix F1":0.888888888888889,"CoSApien BA":0.9205665024630543,"CoSApien F1":0.9090909090909091,"DynaGuardrail BA":0.8977401787426809,"DynaGuardrail F1":0.8919041157847128,"SST unsafe recall":1.0,"HarmBench unsafe recall":0.996875,"Refusal rate":0.004166262959015599,"Cost":1.299847204728224,"Output tokens":292639,"Latency":1.2868233220651746,"Missing":{"Refusal rate":[],"Cost":[],"Output tokens":[],"Latency":[]},"Type":"General-purpose","Local":false,"Open weights":false,"Params (B)":null,"Link":"https://developers.openai.com/api/docs/models/gpt-5.6-terra","Org":"openai"},{"Model":"Claude Opus 5 `reasoning: default (high)`","Avg balanced acc":0.8714355084753209,"Avg F1":0.8548289240559509,"Avg unsafe recall":0.9768749999999999,"Aegis 2.0 BA":0.8411831701980468,"Aegis 2.0 F1":0.8580858085808581,"OpenAI Mod BA":0.8441340655092829,"OpenAI Mod F1":0.8423280423280423,"ToxicChat BA":0.802343131405657,"ToxicChat F1":0.7154213036565978,"WildGuardMix BA":0.8791194756711997,"WildGuardMix F1":0.8641801548205488,"CoSApien BA":0.9427339901477833,"CoSApien F1":0.9341317365269461,"DynaGuardrail BA":0.9190992179199559,"DynaGuardrail F1":0.9148264984227129,"SST unsafe recall":0.96,"HarmBench unsafe recall":0.99375,"Refusal rate":0.03856215482995834,"Cost":6.520677090398218,"Output tokens":390652,"Latency":2.3835540278814733,"Missing":{"Refusal rate":[],"Cost":[],"Output tokens":[],"Latency":[]},"Type":"General-purpose","Local":false,"Open weights":false,"Params (B)":null,"Link":"https://platform.claude.com/docs/en/models/opus-5/overview","Org":"anthropic"},{"Model":"GPT-6.1 Sol `reasoning: default (medium)`","Avg balanced acc":0.8680605970071832,"Avg F1":0.8526558568261545,"Avg unsafe recall":0.970625,"Aegis 2.0 BA":0.8327981499906352,"Aegis 2.0 F1":0.8585720782173715,"OpenAI Mod BA":0.8565691190013303,"OpenAI Mod F1":0.8852459016393441,"ToxicChat BA":0.7964983332261334,"ToxicChat F1":0.6786786786786787,"WildGuardMix BA":0.8889183613321545,"WildGuardMix F1":0.8757062146892656,"CoSApien BA":0.9308292282430213,"CoSApien F1":0.9212121212121211,"DynaGuardrail BA":0.9027503902498246,"DynaGuardrail F1":0.8965201465201466,"SST unsafe recall":0.96,"HarmBench unsafe recall":0.98125,"Refusal rate":0.0013564577075864742,"Cost":1.3069418660982464,"Output tokens":355450,"Latency":2.7703942520311102,"Missing":{"Refusal rate":[],"Cost":[],"Output tokens":[],"Latency":[]},"Type":"General-purpose","Local":false,"Open weights":false,"Params (B)":null,"Link":"https://developers.openai.com/api/docs/models/gpt-6.1-sol","Org":"openai"},{"Model":"ScopeGuard-1 Pro","Avg balanced acc":0.8663792859482794,"Avg F1":0.8470636467940569,"Avg unsafe recall":0.9953125,"Aegis 2.0 BA":0.8507563840371735,"Aegis 2.0 F1":0.8623676612127045,"OpenAI Mod BA":0.8179565014723104,"OpenAI Mod F1":0.8725939505041247,"ToxicChat BA":0.9196372133049142,"ToxicChat F1":0.7906976744186046,"WildGuardMix BA":0.8155509241716138,"WildGuardMix F1":0.7776904948939513,"CoSApien BA":0.9222085385878489,"CoSApien F1":0.9101796407185628,"DynaGuardrail BA":0.8721661541158157,"DynaGuardrail F1":0.8688524590163934,"SST unsafe recall":1.0,"HarmBench unsafe recall":0.990625,"Refusal rate":0.0,"Cost":null,"Output tokens":574122,"Latency":0.4477029349654913,"Missing":{"Refusal rate":[],"Cost":["Aegis 2.0","OpenAI Mod","ToxicChat","WildGuardMix","CoSApien","DynaGuardrail","SST","HarmBench"],"Output tokens":[],"Latency":[]},"Type":"Task-specific","Local":true,"Open weights":false,"Params (B)":null,"Link":"https://principled-intelligence.com/products/orbitals","Org":"principled-intelligence"},{"Model":"GPT-6 Luna `reasoning: default (medium)`","Avg balanced acc":0.8633266220082049,"Avg F1":0.8487421097205855,"Avg unsafe recall":0.9903124999999999,"Aegis 2.0 BA":0.8202563466862118,"Aegis 2.0 F1":0.8516532618409293,"OpenAI Mod BA":0.8498271882851849,"OpenAI Mod F1":0.8865784499054821,"ToxicChat BA":0.8173457596518738,"ToxicChat F1":0.6931818181818181,"WildGuardMix BA":0.8944276030482927,"WildGuardMix F1":0.8825857519788919,"CoSApien BA":0.8983990147783252,"CoSApien F1":0.8834355828220858,"DynaGuardrail BA":0.8997038195993412,"DynaGuardrail F1":0.895017793594306,"SST unsafe recall":0.99,"HarmBench unsafe recall":0.990625,"Refusal rate":0.0,"Cost":0.08547595678713303,"Output tokens":769813,"Latency":2.1124727339483798,"Missing":{"Refusal rate":[],"Cost":[],"Output tokens":[],"Latency":[]},"Type":"General-purpose","Local":false,"Open weights":false,"Params (B)":null,"Link":"https://developers.openai.com/api/docs/models/gpt-6-luna","Org":"openai"},{"Model":"Clef-Flash","Avg balanced acc":0.8613379056424747,"Avg F1":0.8499131309266574,"Avg unsafe recall":0.990625,"Aegis 2.0 BA":0.8274196115283472,"Aegis 2.0 F1":0.8344043499752842,"OpenAI Mod BA":0.8142757256386643,"OpenAI Mod F1":0.8742081447963801,"ToxicChat BA":0.8391152901827796,"ToxicChat F1":0.7363896848137536,"WildGuardMix BA":0.8756838308562447,"WildGuardMix F1":0.8603351955307263,"CoSApien BA":0.9238505747126436,"CoSApien F1":0.9112426035502958,"DynaGuardrail BA":0.8876824009361686,"DynaGuardrail F1":0.8828988068935042,"SST unsafe recall":1.0,"HarmBench unsafe recall":0.98125,"Refusal rate":0.0,"Cost":null,"Output tokens":0,"Latency":0.054611045867204666,"Missing":{"Refusal rate":[],"Cost":["Aegis 2.0","OpenAI Mod","ToxicChat","WildGuardMix","CoSApien","DynaGuardrail","SST","HarmBench"],"Output tokens":[],"Latency":[]},"Type":"General-purpose","Local":true,"Open weights":true,"Params (B)":9,"Link":"https://huggingface.co/Cloudflare/clef-flash","Org":"Cloudflare"},{"Model":"Claude Haiku 5.5 `reasoning: default (medium)`","Avg balanced acc":0.8611574527711358,"Avg F1":0.8495388276345411,"Avg unsafe recall":0.99375,"Aegis 2.0 BA":0.8326774360134723,"Aegis 2.0 F1":0.8494318181818182,"OpenAI Mod BA":0.841394090294121,"OpenAI Mod F1":0.8739172281039461,"ToxicChat BA":0.7846118956419907,"ToxicChat F1":0.6952224052718287,"WildGuardMix BA":0.8864279398762158,"WildGuardMix F1":0.8731034482758621,"CoSApien BA":0.9113300492610837,"CoSApien F1":0.9,"DynaGuardrail BA":0.9105033055399312,"DynaGuardrail F1":0.9055580659737912,"SST unsafe recall":1.0,"HarmBench unsafe recall":0.9875,"Refusal rate":0.005425830830345897,"Cost":0.13903951651971708,"Output tokens":565192,"Latency":1.0177546911872923,"Missing":{"Refusal rate":[],"Cost":[],"Output tokens":[],"Latency":[]},"Type":"General-purpose","Local":false,"Open weights":false,"Params (B)":null,"Link":"https://platform.claude.com/docs/en/models/haiku-5-5/overview","Org":"anthropic"},{"Model":"Claude Sonnet 5.5 `reasoning: default (high)`","Avg balanced acc":0.8606606573692107,"Avg F1":0.8395963561729732,"Avg unsafe recall":0.9653125,"Aegis 2.0 BA":0.8448251596076959,"Aegis 2.0 F1":0.8546426835196889,"OpenAI Mod BA":0.8773207362688586,"OpenAI Mod F1":0.8806584362139918,"ToxicChat BA":0.756208538584207,"ToxicChat F1":0.6518010291595197,"WildGuardMix BA":0.8747175557520385,"WildGuardMix F1":0.8581765557163531,"CoSApien BA":0.9053776683087028,"CoSApien F1":0.8930817610062893,"DynaGuardrail BA":0.9055142856937611,"DynaGuardrail F1":0.8992176714219973,"SST unsafe recall":0.94,"HarmBench unsafe recall":0.990625,"Refusal rate":0.02838872202305978,"Cost":2.600606045925782,"Output tokens":379299,"Latency":1.517854267032817,"Missing":{"Refusal rate":[],"Cost":[],"Output tokens":[],"Latency":[]},"Type":"General-purpose","Local":false,"Open weights":false,"Params (B)":null,"Link":"https://platform.claude.com/docs/en/models/sonnet-5-5/overview","Org":"anthropic"},{"Model":"ScopeGuard 4B-q","Avg balanced acc":0.858808548055651,"Avg F1":0.8522620748937092,"Avg unsafe recall":0.9984375,"Aegis 2.0 BA":0.8443655803852237,"Aegis 2.0 F1":0.8672161172161171,"OpenAI Mod BA":0.7450316631990632,"OpenAI Mod F1":0.8471986417657046,"ToxicChat BA":0.9093903799534679,"ToxicChat F1":0.782608695652174,"WildGuardMix BA":0.8675073330245744,"WildGuardMix F1":0.8519269776876268,"CoSApien BA":0.895320197044335,"CoSApien F1":0.8764044943820225,"DynaGuardrail BA":0.8912361347272417,"DynaGuardrail F1":0.8882175226586102,"SST unsafe recall":1.0,"HarmBench unsafe recall":0.996875,"Refusal rate":0.0,"Cost":null,"Output tokens":447470,"Latency":0.2404087819159031,"Missing":{"Refusal rate":[],"Cost":["Aegis 2.0","OpenAI Mod","ToxicChat","WildGuardMix","CoSApien","DynaGuardrail","SST","HarmBench"],"Output tokens":[],"Latency":[]},"Type":"Task-specific","Local":true,"Open weights":true,"Params (B)":4,"Link":"https://huggingface.co/principled-intelligence/scope-guard-4B-q-2601","Org":"principled-intelligence"},{"Model":"Claude Haiku 4.5","Avg balanced acc":0.856181756254999,"Avg F1":0.8391874463884125,"Avg unsafe recall":0.9778125,"Aegis 2.0 BA":0.8289504596333543,"Aegis 2.0 F1":0.834410740924913,"OpenAI Mod BA":0.848755641961413,"OpenAI Mod F1":0.8704253214638972,"ToxicChat BA":0.7635160611350031,"ToxicChat F1":0.6655290102389079,"WildGuardMix BA":0.8796275244551106,"WildGuardMix F1":0.864321608040201,"CoSApien BA":0.9248768472906403,"CoSApien F1":0.9146341463414636,"DynaGuardrail BA":0.8913640030544726,"DynaGuardrail F1":0.8858038513210927,"SST unsafe recall":0.99,"HarmBench unsafe recall":0.965625,"Refusal rate":0.0,"Cost":0.6990230597810289,"Output tokens":113534,"Latency":0.7457259651273489,"Missing":{"Refusal rate":[],"Cost":[],"Output tokens":[],"Latency":[]},"Type":"General-purpose","Local":false,"Open weights":false,"Params (B)":null,"Link":"https://platform.claude.com/docs/en/models/haiku-4-5/overview","Org":"anthropic"},{"Model":"Nemotron-3.5-Content-Safety","Avg balanced acc":0.8545099908429753,"Avg F1":0.8389705153769803,"Avg unsafe recall":0.9671875,"Aegis 2.0 BA":0.841008865710843,"Aegis 2.0 F1":0.8609948860994886,"OpenAI Mod BA":0.7413508873654171,"OpenAI Mod F1":0.8489932885906041,"ToxicChat BA":0.9030249228715087,"ToxicChat F1":0.7262443438914027,"WildGuardMix BA":0.8619903723351999,"WildGuardMix F1":0.8473034437946718,"CoSApien BA":0.8840311986863711,"CoSApien F1":0.8633879781420766,"DynaGuardrail BA":0.8956536980885121,"DynaGuardrail F1":0.8868991517436381,"SST unsafe recall":1.0,"HarmBench unsafe recall":0.934375,"Refusal rate":0.0,"Cost":null,"Output tokens":85124,"Latency":0.03994821710512042,"Missing":{"Refusal rate":[],"Cost":["Aegis 2.0","OpenAI Mod","ToxicChat","WildGuardMix","CoSApien","DynaGuardrail","SST","HarmBench"],"Output tokens":[],"Latency":[]},"Type":"Task-specific","Local":true,"Open weights":true,"Params (B)":4,"Link":"https://huggingface.co/nvidia/Nemotron-3.5-Content-Safety","Org":"nvidia"},{"Model":"Gemini 4 Argon `reasoning: default (medium)`","Avg balanced acc":0.8536154033804081,"Avg F1":0.8327068296684489,"Avg unsafe recall":0.9393750000000001,"Aegis 2.0 BA":0.8445696573780057,"Aegis 2.0 F1":0.8578267113451411,"OpenAI Mod BA":0.8516093090942165,"OpenAI Mod F1":0.8556593977154725,"ToxicChat BA":0.7698199706789748,"ToxicChat F1":0.6734006734006733,"WildGuardMix BA":0.8376882376882377,"WildGuardMix F1":0.8080495356037151,"CoSApien BA":0.9172824302134648,"CoSApien F1":0.906832298136646,"DynaGuardrail BA":0.9007228152295488,"DynaGuardrail F1":0.8944723618090452,"SST unsafe recall":0.91,"HarmBench unsafe recall":0.96875,"Refusal rate":0.013177017730840035,"Cost":1.7780701482414498,"Output tokens":350704,"Latency":2.8502335948869586,"Missing":{"Refusal rate":[],"Cost":[],"Output tokens":[],"Latency":[]},"Type":"General-purpose","Local":false,"Open weights":false,"Params (B)":null,"Link":"https://ai.google.dev/gemini-api/docs/models/gemini-4-argon","Org":"google"},{"Model":"d1-3B","Avg balanced acc":0.8511494456291162,"Avg F1":0.821425994914243,"Avg unsafe recall":0.9199999999999999,"Aegis 2.0 BA":0.8243281428127547,"Aegis 2.0 F1":0.8418560606060607,"OpenAI Mod BA":0.7610878042679946,"OpenAI Mod F1":0.8340888485947416,"ToxicChat BA":0.9147444612760633,"ToxicChat F1":0.6866866866866866,"WildGuardMix BA":0.8520076347662554,"WildGuardMix F1":0.8398058252427184,"CoSApien BA":0.8694581280788177,"CoSApien F1":0.8478260869565217,"DynaGuardrail BA":0.8852705025728114,"DynaGuardrail F1":0.8782924613987284,"SST unsafe recall":0.99,"HarmBench unsafe recall":0.85,"Refusal rate":0.0,"Cost":null,"Output tokens":0,"Latency":0.03261912474408746,"Missing":{"Refusal rate":[],"Cost":["Aegis 2.0","OpenAI Mod","ToxicChat","WildGuardMix","CoSApien","DynaGuardrail","SST","HarmBench"],"Output tokens":[],"Latency":[]},"Type":"General-purpose","Local":false,"Open weights":true,"Params (B)":3,"Link":"https://huggingface.co/LiquidAI/d1-3B","Org":"LiquidAI"},{"Model":"Gemini 3.8 Flash `reasoning: default (medium)`","Avg balanced acc":0.8503858086737387,"Avg F1":0.8272841619687994,"Avg unsafe recall":0.943125,"Aegis 2.0 BA":0.8462867189724479,"Aegis 2.0 F1":0.8520534388916378,"OpenAI Mod BA":0.8540508430255693,"OpenAI Mod F1":0.8577362409138111,"ToxicChat BA":0.7631384586722144,"ToxicChat F1":0.6689655172413792,"WildGuardMix BA":0.8268100992238923,"WildGuardMix F1":0.7924528301886793,"CoSApien BA":0.9334975369458128,"CoSApien F1":0.9259259259259259,"DynaGuardrail BA":0.8785311952024952,"DynaGuardrail F1":0.8665710186513631,"SST unsafe recall":0.93,"HarmBench unsafe recall":0.95625,"Refusal rate":0.014339695765914156,"Cost":1.0978208022478442,"Output tokens":1982593,"Latency":5.712606979068369,"Missing":{"Refusal rate":[],"Cost":[],"Output tokens":[],"Latency":[]},"Type":"General-purpose","Local":false,"Open weights":false,"Params (B)":null,"Link":"https://ai.google.dev/gemini-api/docs/models/gemini-3.8-flash","Org":"google"},{"Model":"Claude Sonnet 5 `reasoning: default (high)`","Avg balanced acc":0.8501760638656216,"Avg F1":0.8279285188625337,"Avg unsafe recall":0.9671875,"Aegis 2.0 BA":0.829263341601068,"Aegis 2.0 F1":0.8368654509610646,"OpenAI Mod BA":0.8412235524176586,"OpenAI Mod F1":0.8494845360824742,"ToxicChat BA":0.7370722446109863,"ToxicChat F1":0.6197183098591549,"WildGuardMix BA":0.8720790703549324,"WildGuardMix F1":0.8553008595988538,"CoSApien BA":0.9259031198686372,"CoSApien F1":0.9182389937106918,"DynaGuardrail BA":0.8955150543404473,"DynaGuardrail F1":0.8879629629629631,"SST unsafe recall":0.95,"HarmBench unsafe recall":0.984375,"Refusal rate":0.012014339695765914,"Cost":2.295704524755353,"Output tokens":341592,"Latency":1.4464662401005626,"Missing":{"Refusal rate":[],"Cost":[],"Output tokens":[],"Latency":[]},"Type":"General-purpose","Local":false,"Open weights":false,"Params (B)":null,"Link":"https://platform.claude.com/docs/en/models/sonnet-5/overview","Org":"anthropic"},{"Model":"ScopeGuard 4B-g","Avg balanced acc":0.8487574621887627,"Avg F1":0.8213462392913399,"Avg unsafe recall":0.98875,"Aegis 2.0 BA":0.8492076724288193,"Aegis 2.0 F1":0.8661343353687178,"OpenAI Mod BA":0.7858953806973862,"OpenAI Mod F1":0.8167330677290835,"ToxicChat BA":0.8942546759494401,"ToxicChat F1":0.7372175980975031,"WildGuardMix BA":0.8364812709640296,"WildGuardMix F1":0.814968814968815,"CoSApien BA":0.8559113300492611,"CoSApien F1":0.8333333333333334,"DynaGuardrail BA":0.8707944430436397,"DynaGuardrail F1":0.8596902862505866,"SST unsafe recall":0.99,"HarmBench unsafe recall":0.9875,"Refusal rate":0.0,"Cost":null,"Output tokens":569048,"Latency":0.31268355762586,"Missing":{"Refusal rate":[],"Cost":["Aegis 2.0","OpenAI Mod","ToxicChat","WildGuardMix","CoSApien","DynaGuardrail","SST","HarmBench"],"Output tokens":[],"Latency":[]},"Type":"Task-specific","Local":true,"Open weights":true,"Params (B)":4,"Link":"https://huggingface.co/principled-intelligence/scope-guard-4B-g-2601","Org":"principled-intelligence"},{"Model":"Shieldstral-1.0-3B","Avg balanced acc":0.8349757378730879,"Avg F1":0.8328585420402818,"Avg unsafe recall":0.978125,"Aegis 2.0 BA":0.8358419826973024,"Aegis 2.0 F1":0.8626919602529358,"OpenAI Mod BA":0.7323550143820275,"OpenAI Mod F1":0.8462177888611804,"ToxicChat BA":0.9070194135351353,"ToxicChat F1":0.8265582655826558,"WildGuardMix BA":0.9055106451658176,"WildGuardMix F1":0.8951336531871144,"CoSApien BA":0.8148604269293924,"CoSApien F1":0.788888888888889,"DynaGuardrail BA":0.814266944528852,"DynaGuardrail F1":0.7776606954689147,"SST unsafe recall":1.0,"HarmBench unsafe recall":0.95625,"Refusal rate":0.0,"Cost":null,"Output tokens":10321,"Latency":0.011353692039847374,"Missing":{"Refusal rate":[],"Cost":["Aegis 2.0","OpenAI Mod","ToxicChat","WildGuardMix","CoSApien","DynaGuardrail","SST","HarmBench"],"Output tokens":[],"Latency":[]},"Type":"Task-specific","Local":true,"Open weights":true,"Params (B)":3,"Link":"https://huggingface.co/mistralai/Shieldstral-1.0-3B","Org":"mistralai"},{"Model":"Kev 9B","Avg balanced acc":0.7753256431191499,"Avg F1":0.7419679079515774,"Avg unsafe recall":0.9196875,"Aegis 2.0 BA":0.6403481326143183,"Aegis 2.0 F1":0.7482571649883811,"OpenAI Mod BA":0.7456996032152075,"OpenAI Mod F1":0.6766169154228855,"ToxicChat BA":0.6802178450155366,"ToxicChat F1":0.5074626865671642,"WildGuardMix BA":0.7968492554699451,"WildGuardMix F1":0.7497988736926791,"CoSApien BA":0.9027093596059113,"CoSApien F1":0.888888888888889,"DynaGuardrail BA":0.8861296627939804,"DynaGuardrail F1":0.8807829181494661,"SST unsafe recall":0.98,"HarmBench unsafe recall":0.859375,"Refusal rate":0.0,"Cost":null,"Output tokens":543682,"Latency":0.04498844826593995,"Missing":{"Refusal rate":[],"Cost":["Aegis 2.0","OpenAI Mod","ToxicChat","WildGuardMix","CoSApien","DynaGuardrail","SST","HarmBench"],"Output tokens":[],"Latency":[]},"Type":"General-purpose","Local":false,"Open weights":true,"Params (B)":9,"Link":"https://huggingface.co/jaredpalmer/kev-9b","Org":"jaredpalmer"},{"Model":"Kev 27B","Avg balanced acc":0.7655645713900516,"Avg F1":0.707369758423968,"Avg unsafe recall":0.96625,"Aegis 2.0 BA":0.6935867857711242,"Aegis 2.0 F1":0.7848200312989044,"OpenAI Mod BA":0.7841075752924724,"OpenAI Mod F1":0.7365911799761621,"ToxicChat BA":0.589578282923497,"ToxicChat F1":0.3037383177570094,"WildGuardMix BA":0.751996407168821,"WildGuardMix F1":0.6719160104986877,"CoSApien BA":0.8918308702791462,"CoSApien F1":0.8774193548387098,"DynaGuardrail BA":0.8822875069052488,"DynaGuardrail F1":0.8697336561743342,"SST unsafe recall":0.97,"HarmBench unsafe recall":0.9625,"Refusal rate":0.0,"Cost":null,"Output tokens":543696,"Latency":3.9188590687699616,"Missing":{"Refusal rate":[],"Cost":["Aegis 2.0","OpenAI Mod","ToxicChat","WildGuardMix","CoSApien","DynaGuardrail","SST","HarmBench"],"Output tokens":[],"Latency":[]},"Type":"General-purpose","Local":false,"Open weights":true,"Params (B)":27,"Link":"https://huggingface.co/jaredpalmer/kev-27b","Org":"jaredpalmer"},{"Model":"Gliguard-LLMGuardrails-300M","Avg balanced acc":0.7501938481082554,"Avg F1":0.7308126840180617,"Avg unsafe recall":0.9528125000000001,"Aegis 2.0 BA":0.8181116436480089,"Aegis 2.0 F1":0.8446069469835467,"OpenAI Mod BA":0.6589782507361551,"OpenAI Mod F1":0.8074433656957928,"ToxicChat BA":0.8967747981129858,"ToxicChat F1":0.641438032166509,"WildGuardMix BA":0.8741456500077189,"WildGuardMix F1":0.8598130841121496,"CoSApien BA":0.5786124794745484,"CoSApien F1":0.5781990521327014,"DynaGuardrail BA":0.6745402666701148,"DynaGuardrail F1":0.6533756230176709,"SST unsafe recall":0.99,"HarmBench unsafe recall":0.915625,"Refusal rate":0.0,"Cost":null,"Output tokens":null,"Latency":0.014637561049312353,"Missing":{"Refusal rate":[],"Cost":["Aegis 2.0","OpenAI Mod","ToxicChat","WildGuardMix","CoSApien","DynaGuardrail","SST","HarmBench"],"Output tokens":["Aegis 2.0","OpenAI Mod","ToxicChat","WildGuardMix","CoSApien","DynaGuardrail","SST","HarmBench"],"Latency":[]},"Type":"Task-specific","Local":true,"Open weights":true,"Params (B)":0.3,"Link":"https://huggingface.co/fastino/gliguard-LLMGuardrails-300M","Org":"fastino"},{"Model":"Kev 4B","Avg balanced acc":0.6821919607043724,"Avg F1":0.49843078936996993,"Avg unsafe recall":0.861875,"Aegis 2.0 BA":0.6436637070991728,"Aegis 2.0 F1":0.6902248737953189,"OpenAI Mod BA":0.5761025273713292,"OpenAI Mod F1":0.26821192052980136,"ToxicChat BA":0.5234568202434843,"ToxicChat F1":0.09302325581395349,"WildGuardMix BA":0.5343627636731085,"WildGuardMix F1":0.1435523114355231,"CoSApien BA":0.9185139573070608,"CoSApien F1":0.9028571428571429,"DynaGuardrail BA":0.8970519885320788,"DynaGuardrail F1":0.8927152317880794,"SST unsafe recall":0.98,"HarmBench unsafe recall":0.74375,"Refusal rate":0.0,"Cost":null,"Output tokens":543588,"Latency":0.025662255007773638,"Missing":{"Refusal rate":[],"Cost":["Aegis 2.0","OpenAI Mod","ToxicChat","WildGuardMix","CoSApien","DynaGuardrail","SST","HarmBench"],"Output tokens":[],"Latency":[]},"Type":"General-purpose","Local":false,"Open weights":true,"Params (B)":4,"Link":"https://huggingface.co/jaredpalmer/kev-4b","Org":"jaredpalmer"},{"Model":"GLiNER2.5-Decide","Avg balanced acc":0.6703890435977867,"Avg F1":0.6404205673192128,"Avg unsafe recall":0.9515625,"Aegis 2.0 BA":0.6292976611802255,"Aegis 2.0 F1":0.7317460317460317,"OpenAI Mod BA":0.5581448889798424,"OpenAI Mod F1":0.773172569706104,"ToxicChat BA":0.7649432986375261,"ToxicChat F1":0.48841698841698844,"WildGuardMix BA":0.7472822196960128,"WildGuardMix F1":0.7237604636188024,"CoSApien BA":0.6005747126436781,"CoSApien F1":0.48611111111111116,"DynaGuardrail BA":0.7220914804494356,"DynaGuardrail F1":0.6393162393162394,"SST unsafe recall":0.95,"HarmBench unsafe recall":0.953125,"Refusal rate":0.0,"Cost":null,"Output tokens":null,"Latency":0.015771493315696716,"Missing":{"Refusal rate":[],"Cost":["Aegis 2.0","OpenAI Mod","ToxicChat","WildGuardMix","CoSApien","DynaGuardrail","SST","HarmBench"],"Output tokens":["Aegis 2.0","OpenAI Mod","ToxicChat","WildGuardMix","CoSApien","DynaGuardrail","SST","HarmBench"],"Latency":[]},"Type":"General-purpose","Local":true,"Open weights":true,"Params (B)":0.5,"Link":"https://huggingface.co/fastino/GLiNER2.5-Decide","Org":"fastino"},{"Model":"Parakh 4B","Avg balanced acc":0.6424026689270513,"Avg F1":0.4157549620082835,"Avg unsafe recall":0.8465625,"Aegis 2.0 BA":0.48570865600414004,"Aegis 2.0 F1":0.4796663190823775,"OpenAI Mod BA":0.5038314176245211,"OpenAI Mod F1":0.01520912547528517,"ToxicChat BA":0.5181326809663961,"ToxicChat F1":0.07329842931937172,"WildGuardMix BA":0.5360770774563878,"WildGuardMix F1":0.13793103448275862,"CoSApien BA":0.9078407224958949,"CoSApien F1":0.8877005347593583,"DynaGuardrail BA":0.9028254590149685,"DynaGuardrail F1":0.9007243289305497,"SST unsafe recall":0.99,"HarmBench unsafe recall":0.703125,"Refusal rate":0.0,"Cost":null,"Output tokens":543536,"Latency":0.08423822792246938,"Missing":{"Refusal rate":[],"Cost":["Aegis 2.0","OpenAI Mod","ToxicChat","WildGuardMix","CoSApien","DynaGuardrail","SST","HarmBench"],"Output tokens":[],"Latency":[]},"Type":"General-purpose","Local":false,"Open weights":true,"Params (B)":4,"Link":"https://huggingface.co/YAlgoG/parakh-4b","Org":"YAlgoG"},{"Model":"Laya","Avg balanced acc":0.5967944440162024,"Avg F1":0.4771753332143125,"Avg unsafe recall":0.6571875,"Aegis 2.0 BA":0.5986087037484126,"Aegis 2.0 F1":0.46336633663366333,"OpenAI Mod BA":0.6745057243880532,"OpenAI Mod F1":0.7395934172313648,"ToxicChat BA":0.6230518263538795,"ToxicChat F1":0.3131407269338304,"WildGuardMix BA":0.5659242417863107,"WildGuardMix F1":0.34782608695652173,"CoSApien BA":0.5812807881773399,"CoSApien F1":0.49367088607594933,"DynaGuardrail BA":0.5373953796432186,"DynaGuardrail F1":0.5054545454545455,"SST unsafe recall":0.83,"HarmBench unsafe recall":0.484375,"Refusal rate":0.0,"Cost":null,"Output tokens":0,"Latency":0.01968592405319214,"Missing":{"Refusal rate":[],"Cost":["Aegis 2.0","OpenAI Mod","ToxicChat","WildGuardMix","CoSApien","DynaGuardrail","SST","HarmBench"],"Output tokens":[],"Latency":[]},"Type":"General-purpose","Local":true,"Open weights":true,"Params (B)":0.4,"Link":"https://huggingface.co/convaiinnovations/laya","Org":"convaiinnovations"},{"Model":"Kev 0.8B","Avg balanced acc":0.5283772838797435,"Avg F1":0.4387165100572181,"Avg unsafe recall":0.6950000000000001,"Aegis 2.0 BA":0.4161449260613357,"Aegis 2.0 F1":0.6045824094604583,"OpenAI Mod BA":0.6168980297190673,"OpenAI Mod F1":0.4932975871313672,"ToxicChat BA":0.5510994275524485,"ToxicChat F1":0.22645290581162325,"WildGuardMix BA":0.5051780275918207,"WildGuardMix F1":0.02860858257477243,"CoSApien BA":0.5732758620689655,"CoSApien F1":0.6292134831460674,"DynaGuardrail BA":0.5076674302848231,"DynaGuardrail F1":0.6501440922190203,"SST unsafe recall":0.89,"HarmBench unsafe recall":0.5,"Refusal rate":0.0,"Cost":null,"Output tokens":543432,"Latency":0.009999724105000496,"Missing":{"Refusal rate":[],"Cost":["Aegis 2.0","OpenAI Mod","ToxicChat","WildGuardMix","CoSApien","DynaGuardrail","SST","HarmBench"],"Output tokens":[],"Latency":[]},"Type":"General-purpose","Local":false,"Open weights":true,"Params (B)":0.8,"Link":"https://huggingface.co/jaredpalmer/kev-0.8b","Org":"jaredpalmer"},{"Model":"PolyGuard-Qwen","Avg balanced acc":null,"Avg F1":null,"Avg unsafe recall":0.985625,"Aegis 2.0 BA":0.8470965311241774,"Aegis 2.0 F1":0.8693259972489684,"OpenAI Mod BA":0.7156025103175415,"OpenAI Mod F1":0.8373245251857967,"ToxicChat BA":0.9201878142528572,"ToxicChat F1":0.7095435684647302,"WildGuardMix BA":0.900085610430438,"WildGuardMix F1":0.8888888888888888,"CoSApien BA":null,"CoSApien F1":null,"DynaGuardrail BA":null,"DynaGuardrail F1":null,"SST unsafe recall":0.99,"HarmBench unsafe recall":0.98125,"Refusal rate":0.0,"Cost":null,"Output tokens":249975,"Latency":0.3145637270063162,"Missing":{"Refusal rate":["CoSApien","DynaGuardrail"],"Cost":["Aegis 2.0","OpenAI Mod","ToxicChat","WildGuardMix","CoSApien","DynaGuardrail","SST","HarmBench"],"Output tokens":["CoSApien","DynaGuardrail"],"Latency":["CoSApien","DynaGuardrail"]},"Type":"Task-specific","Local":true,"Open weights":true,"Params (B)":8,"Link":"https://huggingface.co/ToxicityPrompts/PolyGuard-Qwen","Org":"ToxicityPrompts"},{"Model":"Qwen3Guard-Gen-0.6B","Avg balanced acc":null,"Avg F1":null,"Avg unsafe recall":0.9728125,"Aegis 2.0 BA":0.8143705929925266,"Aegis 2.0 F1":0.8521587440034889,"OpenAI Mod BA":0.6587906590720465,"OpenAI Mod F1":0.8105939004815408,"ToxicChat BA":0.9103385447278711,"ToxicChat F1":0.6476014760147603,"WildGuardMix BA":0.8965019016743154,"WildGuardMix F1":0.8847736625514404,"CoSApien BA":null,"CoSApien F1":null,"DynaGuardrail BA":null,"DynaGuardrail F1":null,"SST unsafe recall":0.98,"HarmBench unsafe recall":0.965625,"Refusal rate":0.0,"Cost":null,"Output tokens":74796,"Latency":0.02079308172687888,"Missing":{"Refusal rate":["CoSApien","DynaGuardrail"],"Cost":["Aegis 2.0","OpenAI Mod","ToxicChat","WildGuardMix","CoSApien","DynaGuardrail","SST","HarmBench"],"Output tokens":["CoSApien","DynaGuardrail"],"Latency":["CoSApien","DynaGuardrail"]},"Type":"Task-specific","Local":true,"Open weights":true,"Params (B)":0.6,"Link":"https://huggingface.co/Qwen/Qwen3Guard-Gen-0.6B","Org":"Qwen"},{"Model":"Qwen3Guard-Gen-4B","Avg balanced acc":null,"Avg F1":null,"Avg unsafe recall":0.9903124999999999,"Aegis 2.0 BA":0.8262568598559443,"Aegis 2.0 F1":0.8611476127901883,"OpenAI Mod BA":0.6857413281489819,"OpenAI Mod F1":0.8267526188557615,"ToxicChat BA":0.9257464995530873,"ToxicChat F1":0.6901270772238514,"WildGuardMix BA":0.9027311130759407,"WildGuardMix F1":0.8919288645690834,"CoSApien BA":null,"CoSApien F1":null,"DynaGuardrail BA":null,"DynaGuardrail F1":null,"SST unsafe recall":0.99,"HarmBench unsafe recall":0.990625,"Refusal rate":0.0,"Cost":null,"Output tokens":74490,"Latency":0.06245130905881524,"Missing":{"Refusal rate":["CoSApien","DynaGuardrail"],"Cost":["Aegis 2.0","OpenAI Mod","ToxicChat","WildGuardMix","CoSApien","DynaGuardrail","SST","HarmBench"],"Output tokens":["CoSApien","DynaGuardrail"],"Latency":["CoSApien","DynaGuardrail"]},"Type":"Task-specific","Local":true,"Open weights":true,"Params (B)":4,"Link":"https://huggingface.co/Qwen/Qwen3Guard-Gen-4B","Org":"Qwen"},{"Model":"Qwen3Guard-Gen-8B","Avg balanced acc":null,"Avg F1":null,"Avg unsafe recall":0.9918750000000001,"Aegis 2.0 BA":0.8307503429251324,"Aegis 2.0 F1":0.8653846153846154,"OpenAI Mod BA":0.6838256193367214,"OpenAI Mod F1":0.824858757062147,"ToxicChat BA":0.9253212116104162,"ToxicChat F1":0.685382381413359,"WildGuardMix BA":0.9073730228902643,"WildGuardMix F1":0.8972089857045609,"CoSApien BA":null,"CoSApien F1":null,"DynaGuardrail BA":null,"DynaGuardrail F1":null,"SST unsafe recall":0.99,"HarmBench unsafe recall":0.99375,"Refusal rate":0.0,"Cost":null,"Output tokens":74690,"Latency":0.10168820386752486,"Missing":{"Refusal rate":["CoSApien","DynaGuardrail"],"Cost":["Aegis 2.0","OpenAI Mod","ToxicChat","WildGuardMix","CoSApien","DynaGuardrail","SST","HarmBench"],"Output tokens":["CoSApien","DynaGuardrail"],"Latency":["CoSApien","DynaGuardrail"]},"Type":"Task-specific","Local":true,"Open weights":true,"Params (B)":8,"Link":"https://huggingface.co/Qwen/Qwen3Guard-Gen-8B","Org":"Qwen"}],"updated_at":"2026-10-14T13:15:40+00:00","error":null}