Sudan's leadership bid drew criticism from rights groups that say a Sudanese presidency would hurt AU efforts to improve Africa's record on democracy and human rights. Several African regional blocs also oppose it.
Mustafa Osman Ismail, Sudan's presidential adviser on foreign affairs, said on Monday: "We do not want to make any division in order to achieve an objective, so if that means that Sudan should withdraw, we will withdraw."
Ismail's comments followed several hours of talks among heads of state.
Sudan nominated itself to chair the 53-member AU, based on a tradition that the host of its summit becomes the organisation's next leader.
Sudan, which is under fire for rights abuses, wants to succeed Nigeria at the two-day summit that opened in Khartoum on Monday.
"The general consensus is Nigeria [will stay on]. It will be for 12 months," a delegate, who asked not to be named, said as the private meeting of heads of state and ministers broke up.
Earlier an AU official and delegates said five African states had asked Sudan on Sunday to withdraw.