Hornqvist assisted on Letang's goal early in the first, and then made 2-0 later in the period.
Hornqvist was in the lineup for the first time since undergoing offseason surgery on a broken finger. The Penguins (2-1-1) have killed off 17 consecutive power plays.
"We had a good game plan going in", Letang said of the penalty-killing success.
Capitals forward Christian Djoos scored in his NHL debut and Alex Ovechkin notched his league-leading eighth goal of the season.
"You can see the instincts are really good", Trotz said of Djoos, Washington's seventh-round pick in the 2012 draft. "I was going to play good and play a structured game and detailed game, but happy with the goal and the assist". His IQ is really good. It's one game, and we'll be patient.
With Washington trailing 3-1 in the third, Djoos avoided a Pittsburgh player and slid a pass into the slot that Ovechkin redirected past Murray with 7:09 left to play. Holtby made 33 saves for Washington (2-1-1).
Pittsburgh's first two goals came with five seconds and six seconds to go on those power plays.
The Penguins doubled their lead on a similar scenario at 8:20 of the second.
"That's usually where I end up being and where I get my points and like to be", Hornqvist said of his work in front of the net. Capitals G Braden Holtby, who won his first two starts of the season, missed winning each of his first three for the first time in his National Hockey League career. Nicklas Backstrom also had a goal, while dishing out two assists.
Capitals defenseman Christian Djoos scored in his National Hockey League debut with 53 seconds left in the second period to make it 2-1. The game was tied 1-1 until Nail Yakupov, Andrighetto and Matt Duchene scored in the second period to help the Avs complete a season sweep of Boston.
Sven Andrighetto scored twice and Alexander Kerfoot had his first National Hockey League goal in helping the Avalanche double up the Bruins, 6-3. Sullivan said there's no definitive timeline for the return of defenseman Ian Cole, who took a puck to the face Saturday and was placed on injured reserve.