I posed a question a few days ago: Was Adam with Eve when she was talking with the serpent? (See post and comments.)
As promised, I consulted my shelf of scholars.
They were not especially helpful. Most don't address the question but seem to assume that Adam was absent. The one who does address the question concludes that he was present.
I still lean in that direction as well, though I think the text is just vague enough to allow either interpretation.