Yoshitaka Sakurada, 68, revealed that he has no use for the devices and that he's delegated such work to his secretaries since he started running his own business when he was 25, the New York Times reported.
The minister in charge of cybersecurity said he doesn't use computers.
"It's shocking to me that someone who hasn't even touched computers is responsible for dealing with cybersecurity policies", Imai said.
When asked about the power grid and malware, Sakurada said USB was "basically never used" in the utility systems, appearing to not know what it might be.
"Doesn't he feel ashamed?" wrote one Twitter user.
"Today any company president uses a PC. Indeed it might be the strongest kind of security!"
Although the minister is not expected to have much hands-on responsibility in the handling of either cybersecurity or the Olympics, Sakurada's high-profile bungling is an embarrassment for Abe.
Ministers in Japan nearly always get parliamentary questions in advance.
Sakurada's responses in parliament and news conferences have drawn criticism before. He previously struggled to answer simple questions about the Tokyo 2020 Olympic and Paralympic Games.
He also denied knowledge that the president of the International Olympic Committee had asked North Korean leader Kim Jong-un in March to permit his country's athletes to compete.
"This is not something I should be meddling in in my capacity", he said, according to the Asahi newspaper.
Sakurada blamed one particularly unimpressive performance in parliament on the opposition MP Renho Murata, complaining that she had not given him her questions in advance.