Getting down to business means getting down to the nitty–gritty (1963) — "the heart of the matter," "the essential facts."
No messing around. Let's get right to it.
Nits are the eggs or larvae of insects such as lice. Being almost microscopic, they are extremely difficult to spot. Getting down to them means you are indeed attending to the smallest of details. To be a successful nitpicker, you must be obsessive about the smallest of details and the pettiest of matters.
Grit initially referred to minute particles of sand and stone. To determine the texture (coarseness or fineness) or firmness of a rock — its basic nature — you had to examine the smallest particles — getting down to the grit. Those stones judged "of good hard quality" were considered clear grit.
It wasn't easy sifting through the grit. It took character to grit it out — "to endure the most difficult and trying situation." To be the grit (19th C.) was to have genuine spirit or "pluck", be the "right" sort of person, the "genuine article" who would see things through to the end, no matter how difficult, as with the noble John Wayne in the film "True Grit"
Having grit today primarily pertains to having perseverance and passion for long-term goals while not being troubled by the details and hard work necessary along the way.
Attention to detail is not without its downside. Carrying things to an extreme can leave you overly meticulous. It's the Latin metus, "fear" + the diminutive ul+ osus, "full of," leaving you "full of little fears." Could it be that many of your actions — done on behalf of a noble calling — are, at bottom, just fear–driven?
Worse yet, is being overly fastidious, from the Latin fastidiosus, "disdainful" or "squeamish", from fastus, "contempt'+ taedium, "aversion," making your actions tedious to others, "wearisome to the point of being disgusting."
Like the nit (1902–03), which once described "an inconsequential or obnoxious person," you too could then easily become a target to be picked on.