Karlstrom, who joined Skanska in 1983 and has been chief executive since 2008, said his decision to leave was not based on any disagreements with the board about strategy.
"I will have done exactly 10 years as CEO. I think that's enough, really. We are entirely in agreement on the direction of the company," he said.
Karlstrom, 61, said he would resign from the Skanska board at its April annual shareholder meeting but would stay on as a senior adviser until the end of January 2019, the resignation date stipulated in his contract.
Skanska, which built the Gherkin in London, said on Wednesday it aimed to name Karlstrom's successor by the end of this year.
Chairman Hans Biorck told Reuters that although Skanska has a succession program, Karlstrom's resignation came earlier than expected, which is why a new CEO would be appointed only in the coming months.
"Johan's contract said he'd resign in January 2019. He has at his own initiative wanted to bring it forward by nine months, so the matter is brought to the fore earlier than we had planned," Biorck said, adding:
"I believe he felt it would be nice with some more time for himself, and I have full respect for the fact it's a tremendously intense job."
Skanska, whose biggest market is the United States, reported a fall in second-quarter profit after writedowns on its U.S. civil operations and British projects, though orders and sales growth both beat expectations.
Skanska shares were down 1.1 percent at 1435 GMT as the Stockholm bourse <.OMXSPI> was flat, taking the stock's year-to-date fall to 16.5 percent.
Besides Sweden, Norway, Finland, the United States and Britain, Skanska also has operations in the Czech Republic, Slovakia and Poland.
Karlstrom said he had no plans currently to take on a new operational role after he leaves Skanska.
(Reporting by Anna Ringstrom; editing by David Clarke and Elaine Hardcastle)
By Anna Ringstrom