It depends on what your contract says and what you want to do. Ultimately you need to decide what you want and then we can work backwards from that. But no just having the conversation does not automatically oust you.

What about other clauses in the contract? Usually such agreements included provisions regarding scope of work, termination etc.

Did he give you any reason for the change?

Yes, what's in your agreement is very important.

For sure, I'd send him/her a bill for the work you've done.

And write a Lesson's Learned report on the whole issue.

One of the important lesson's I learned from such a situation early on in my career, was to create my own agreement and have a lawyer look at it for it's legal issues (to protect me).

To answer your first question: Yes but it doesn't matter. You did say that if he followed this route, you'd look at exiting. Presumably, he's following this route, so presumably, you're looking at exiting. But that doesn't matter. Looking at exiting isn't the same as exiting. He is not entitled to say that you have exited until you exit.

Next, you mention you signed an IP agreement, so you do not own the IP. This is normal. But what did you get in return? You should talk to your lawyer about the validity of your IP agreement if you got nothing in return.

Bottom line is that this is an area for a lawyer. If your interest in the company is not worth a $1000 expense talking to (and having a couple letters to the company written by) your lawyer, then it's time to move on. But if it is important enough, the lawyer should be your next "tool" to use.

Before using that tool, you may want to make it known to the other guy that you have "made an appointment my your lawyer next Tuesday, but would like a chance to meet and settle the matter person to person before that time." Sometimes, just the knowledge that lawyers are getting involved is enough to bring people to the table. Sometimes not.

Most of this is related to legal issues which I doubt many will want to answer. See legal advice.
On the sanity side, are you losing any money? Is the company making money?
Sometimes it is better to move on to other projects unless you were well protected by a contract.
Don't stop taking massive action.
Best of Luck,
Michael T. Irvin
michaelirvin.net
My books are available exclusively through Amazon Books. Check out my book "Copywriting Blackbook of Secrets"
Copywriting, Startups, Internet Entrepreneur, Online Marketing, Making Money