Yoshiro Mori, who suggested in an online meeting of the Japanese Olympic Committee that women talk too much in meetings, "stoked a social media backlash", and on Twitter "users quickly began calling for Mori to resign", the US daily said on its online edition.
Mori plans to speak to media at 2 p.m.to explain his comments and apologize, Nippon TV reported.
However, as of November, there were just five women among the board's 24 members.
"Last night, my wife gave me a thorough scolding", Mori was quoted by the Mainichi as saying. "She said: 'You've said something bad again, haven't you? I'm going to have to suffer again because you've antagonized women.' This morning, my daughter and granddaughter scolded me as well".
His latest comments triggered anger on social media, with the hashtag #Moriresign trending on Twitter.
In an apparent protest of Mr Mori's comments, Noriko Mizoguchi, a former judo Olympic silver medallist, tweeted the International Olympic Committee's code of ethics and said that any type of harassment should be rejected.
Mori's controversial sexism remarks come with increased scrutiny on organisers with the authority insisting that the pandemic-postponed Tokyo Olympics can go ahead despite surging virus cases in the country and in parts of the world.
He reportedly said "board meetings with lots of women take longer" because "women are competitive - if one member raises their hand to speak, others might think they need to talk too", according to reports in the Japanese press.
According to the report the comments were met with some laughter.
Renho, a prominent opposition politician, called Mr Mori's remarks "shameful". "Everyone ends up saying something".
Almost 80 per cent of the Japanese public opposes holding the Games as scheduled in July, according to the most recent poll. "If you want to increase female membership, you would be in trouble unless you put time limits in place", he is reported to have added.
U.S. and European media have reported on questionable remarks made by the head of the organizing committee of the Tokyo Olympics and Paralympics. "But a board meeting with plenty of women will make it drag on", Mori said.
Earlier, Japan Olympic Committee director Kaori Yamaguchi said the comments were "unfortunate" considering they were coming from the president of the organising committee.