I would fix it earlier than later, depending on the cause it may be damaging other parts.
I just rebuilt on that lost some parts and only driven a few miles after issues started and you could see where the gears had rubbed together.
This is what I found in the gearbox when I stripped it
Paul