Taking care of our independent-minded elderly parents can be a huge challenge.

That challenge will increase as parents grow older. Between 7 million and 10 million adults are caring for aging parents from a long distance, according to Pew Research Center.

Meanwhile, financial fraud against older Americans costs them as much as $36.5 billion a year, the Consumer Financial Protection Bureau says.

So a good question is: How do you know it is time to step in to protect your parents from financial predators or even from themselves?

"There are two decisions that become particularly difficult as our loved ones age," says Sen. Susan Collins, R, of Maine, head of the Senate Committee on Aging. "One is deciding when it is no longer safe for them to be driving. The second is when it is no longer prudent for them to handling their own financial affairs. Both are difficult decisions."

Knowing when to step in can be emotional and complicated. But financial planners say there are telltale signs.

"More often than not, it's the kids coming to us asking for help," says Ryan Wibberley, CEO of CIC Wealth in Gaithersburg. "They may not want to have the conversations with parents. Elderly parents are reluctant to tell them about finances."

Financial adviser Tom West, a partner at SEIA in Tysons Corner, says: "The biggest challenges of adult children and aging parents usually have to do with the ideas of privacy, control and independence - 'I don't want you to know my stuff.' It may be because of embarrassment."

West adds, "I've met a lot of aging parents that would prefer to suffer mildly than to burden the kids."

Collins says there are a number of warning signs. "One is failing to pay bills on time and confusion about whether or not bills have been paid," she says. "Indications that he or she can no longer manage a checking account."

Another, she says, "is if you are seeing unusual withdrawals that are not tied to a purchase, like a washing machine or a new car."

Leslie Thompson, partner at Spectrum Management in Indianapolis, cites others.

"If we notice things are different, like forgetfulness, bouncing checks or needing money outside of the norm, we will have conversations with our clients," she says. "If we feel the client is in danger, we will contact the family."

Michelle Jacko, CEO of Jacko Law Group in San Diego, says one longtime client missed an appointment with a financial adviser, who then made a home appointment. The adviser discovered that the client was disheveled. A man half her age was living there and ordered the adviser to leave. The adviser reported the situation to financial regulators and law enforcement officials, and the woman is now getting help.

Jacko says people can take steps to prepare: Organize financial documents and create a durable power of attorney. Also, she says, interview your parents' financial advisers and get to know what your parents want.

Collins has introduced a bill, the Senior Safe Act, which would empower banks and credit unions to combat financial abuse.

"It's similar to a law we have in Maine that has been valuable," she says. In one case, a teller noticed a customer was suddenly making large withdrawals. The manager notified family members, who discovered she was a victim of a scam.

"In the bill, we say that if the front-line workers at financial institutions, like bank or credit unions, are trained to recognize these signs and report their concerns in good faith to the authorities, they are not violating bank secrecy laws," she says. "We have been able to stop some truly sad situations."