STRAIGHT TO THE POINT – SHORT ANSWER –
The baby seems happy/comfortable.
The baby seems to be happy/comfortable.
They both are correct, and there is no difference in meaning.
DIGGING DEEPER – EXPLANATION –
We will discuss here this structure – X copular verb Y.
At this very beginning we will discuss two types of Copular Clauses – one is Ascriptive and the other is Specifying.
1. Sam [THEME] was hungry [PROPERTY].
2. Robin [VARIABLE] was the tallest of them [VALUE].
In Ascriptive use, as in sentence #1 the Predicative Complement (PC) ascribes the property or the characteristics of the subject. On the other hand in the Specifying use, as in sentence #2, the PC helps to specify or identify the subject.
At times they are ambiguous, that means they can be explained both as Ascriptive use and Specifying use.
The victim is Kim’s sister.
In specifying use, the victim is identified as Kim’s sister. In ascriptive use, it only says the victim is the sister of Kim, now imagine that Kim has more than one sister,it doesn’t say which sister of Kim is the victim. So it’s in that situation an ascriptive use. (In specifying use it implies that Kim has only one sister.)
The verb – be – can license both ascriptive PC and specifying PC, whereas other copular verbs can license only the ascriptive PC. If other copular verbs license to-infinitive clause the limitation can be overcome with the use of to be before a Specifying PC.
It was Max. [The verb – be – can take a specifying PC]
!It seemed Max [INCORRECT – because the copular verb – seem – can’t take a specifying PC]
And it can be overcome with the use of to be –
It seemed to be Max.
But the use of to be after seem is not restricted to only the situation where we need a specifying PC. As seem can license a to-infinitive, we can use optional to be even before an ascriptive PC. But notice the difference, in the ascriptive use, the use of to be is optional, but in case of specifying use the use of to be is obligatory.
In OP’s examples the AdjPs – happy or comfortable – denotes a property, and nothing but Ascriptive PCs in those sentences. So the use of to be after the verb – seem – is optional.
Now that we know when to use to be after seem optionally and when to use it obligatorily, let’s focus on what extra meaning to be adds, if at all.
Most of the times there is no difference in meaning between seem to be and seem, but seem to be is preferred when we mean something that appears to be definitely true (objective facts). On the other hand seem without to be is preferred when it’s based on personal feeling (subjective impression).
For example –
The doctors have done all the tests, and he definitely seems to be insane.
It seems insane, but I think I’m in love with the postman. [NOT
It seems to be insane]