They held assembly members at gunpoint and demanded they back down from a recently approved draft constitution.
It is unclear whether the fighters are still in the assembly.
The draft calls for a presidential and general election no more than 180 days from the passing of a constitution.
It is understood that the draft bans Haftar from running as president.
The United Nations Support Mission in Libya expressed concern over the reports of attack.
“Disturbed by reports of attacks on the constitutional drafting assembly (CDA) HQ in Bayda, Libya,” the mission tweeted on Saturday.
It added that “as an independent elected body, the CDA must be allowed to work without intimidation or interference”.
“We commit to a ceasefire and to refrain from any use of armed force for any purpose that does not strictly constitute counterterrorism,” Serraj and Haftar said in a joint declaration.
The joint declaration also said the two main rivals will work on holding early presidential and parliamentary elections.
Shortly after the ceasefire announcement, Hafter told Saudi daily Al Awsat News that not everything in the Paris agreement can be implemented.
Tuesday’s meeting was the first between the Libyan factions since exploratory talks hosted by the United Arab Emirates in Abu Dhabi in early May.
Libya has been locked in a state of violence and turmoil since 2011, when a bloody popular uprising ended with the overthrow and death of former President Muammar Gaddafi.
The country has splintered into rival political and armed groups, with the factions backing opposing governments and parliaments in the east and the west.