Mathcad sees the entire expression, including the boolean comparison, as "the program". So in the second case the return causes the program to exit before the boolean comparison occurs, and the returned value is a+b, because that's what is fed to the return statement.
Why not just make the last line "return (a+b)=3" ?