bag update - 24Q4

[leondz]

probes

• include latentinjection
• include suffix
• include tap cached
• include autodan cached
• include ansiescape
• remove dans except daninthewild,danablation
• include all encoding
• ensure new packagehallucination items are there
• move knownbadsignatures to av_spam_scanning
• remove donotanswer
• enable extended detectors
• consider only using tier1,2 probes?

models

• nous?
• long context
• jamba
• palmyra

[leondz]

tier 1 + 2 probes:

plugins:
    probe_spec: atkgen.Tox,continuation.ContinueSlursReclaimedSlurs,dan.Ablation_Dan_11_0,dan.AutoDANCached,dan.DanInTheWild,encoding,fileformats.HF_Files,goodside.Tag,goodside.ThreatenJSON,goodside.WhoIsRiley,grandma,latentinjection,leakreplay,lmrc.Bullying,lmrc.Deadnaming,lmrc.QuackMedicine,lmrc.SexualContent,lmrc.Sexualisation,lmrc.SlurUsage,malwaregen,misleading.FalseAssertion50,packagehallucination,promptinject,realtoxicityprompts.RTPBlank,replay.Repeat,snowball.GraphConnectivityMini,snowball.SenatorsMini,suffix.GCGCached,tap.TAPCached,xss.MarkdownImageExfil
    extended_detectors: true

• ensure mini and non-mini versions are all accounted for
• do we want to add.. BEAST? AutoDAN? TAP? GCG? yeah I guess, what's fast

[erickgalinkin]

IMO, we should run everything Tier 1 + Tier 2 that is active.
I'd ideally like to have a bag of ~20 models but that seems steep.