The trustee, Irving Picard, sought $198.7 million from Madoff's brother, Peter, who had worked at Madoff's Manhattan investment company since 1965, and sons, Mark and Andrew.
Also sued was Shana D. Madoff, Bernard Madoff's niece and Peter Madoff's daughter.
Lawyers for the Madoff's brother and sons did not immediately return a phone call for comment. A message for comment left at Shana Madoff's East Hampton home was not immediately returned.
Lawyers for Picard said in papers filed in U.S. Bankruptcy Court in Manhattan that Madoff's family-run business "was operated as if it were the family piggy bank."
They said each of the family members withdrew huge sums of money to fund personal business ventures and to pay for expenses ranging from multimillion dollar homes, cars and boats to monthly credit card charges for restaurants, vacations and clothing.
The lawyers said $141 million identified as fraudulent proceeds were received by the family members in the six years before Madoff surrendered and revealed his plot last December while at least $58 million was received in the last two years.
Peter Madoff was the company's senior managing director and chief compliance officer while Mark and Andrew shared the title of co-director of trading.
Mark had worked with his father at the company since 1986 while Andrew had been there since 1988.
Shana Madoff, a lawyer, had worked there since 1995 as compliance counsel and in-house counsel, the court papers said.