anything can help, like a class of usage and examples
Here are few examples that i know,
1. SegWit address with uncompressed public key,
https://bitcointalk.org/index.php?topic=5192454.0.
2. Any transaction with multiple OP_RETURN or OP_RETURN with pushed data more than 80 bytes,
https://bitcointalk.org/index.php?topic=5275615.0.
3. Ordinal NFT with TX size 3915537 bytes which mined by Luxor,
https://ordinals.com/inscription/0301e0480b374b32851a9462db29dc19fe830a7f7d7a88b81612b9d42099c0aei0. Take note the TX size makes it non-standard.
any machine learning model that i can use when studying them ( i have a file containing the hash of all non-standard transactions and the script associated)
AI/ML is wrong approach in this case when you could just look for rules of standard transaction and create script to identify transaction which doesn't match the rule. For current rule, you could check Bitcoin Core source code at
https://github.com/bitcoin/bitcoin/tree/v24.0.1/src/policy.
also i see a lot of them recently as an output for coinbase TX, what is that about ?
Any example? I don't know which one you're talking about.