I've always thought about it like that, but with a little twist. You coming to the past in and of itself creates an entirely new timeline. This new timeline is not, however, the timeline that you are familiar with. Since there is now a different timeline, there is now a different "you." That "you" did not have the same experiences in their life that led to building the time machine that you had in your past. Since the new "you" had different memories and experiences throughout their life, they are no longer the "you" that you are familiar with. That means that the "you" that built the time machine never existed. If the "you" that used to exist never built the time machine, you never went back in time. Welp, back to square 1.
