Loading…
29
Why does nobody admit the small 3B model beat the big one for my ticket sorting
I tried running a 70B model on my shop laptop in Boise to sort 400 work tickets and it kept timing out and making stuff up. Then I switched to a small 3B model that runs local and it nailed the categories in 12 minutes flat, no internet needed. Is it just me or are people sleeping on small local models for boring real work like this?
1 comments
Log in to join the discussion
Log In1 Comment
mark_carr718d ago
nailed the categories in 12 minutes flat" sounds great until you ask what happens when the tickets don't fit the boxes. Small models are fast because they mostly pattern match, and for sorting 400 tickets with clear labels that works fine. Throw in a vague ticket like "customer says thing broke again, maybe shipping or maybe wrong part" and the 3B will just pick whatever it saw most in training. I ran a 3B on support tags once and it lumped refund requests and tech issues together because they both had the word "broken." The 70B was slow but it at least asked itself if the tag made sense. Speed is not the same as getting it right.
10