Backwards-compatible wire protocols require a great deal of parsing overhead so that new messages can be sent without breaking old parsers; good examples are HTTP, Protocol Buffers, and FIX. A popular approach is to send each message as a sequence of self-describing tag-value pairs; this spends a great deal of space and time on metadata about the format of the message stream.
By sacrificing backwards-compatibility, we can instead create optimized binary wire protocols which support exactly and only the current set of features.
A wire protocol that is not backwards-compatible is still useful; it just means that whenever the service changes the wire protocol, all the clients must be upgraded. (Translation layers can be installed between the client and the service, but that sacrifices performance, which defeats our original goal.)
Upgrading in sync is straightforward if the service and the clients are controlled by the same entity. However, if the service and clients are controlled by different entities, such synchronized upgrades become harder. For example, TCP/IP and Ethernet have so many deployed implementations that they can essentially never be upgraded.
We can ensure that the service and clients are controlled by the same entity by having user code download a backwards-compatible client library on the fly, instead of deploying it as part of their own code.
Users program against an interface definition for the client library. Then, at runtime, some minimal bootstrap code downloads the current version of the client library from the service and runs it inside a sandbox.
This is what web browsers do. A web browser contacts a server to download HTML and Javascript; that HTML and Javascript then further interacts with the server, using an arbitrary private protocol, which might not even be HTTP-based, to implement some web application.
Both for client libraries and the web, this allows the service provider to ignore backwards compatibility issues in the wire protocol, since the client library speaking the client-side of the protocol will always be up to date.
For this to work, each new version of the client library must be backwards-compatible, since users may have written code with old interface definitions. This is much simpler to do while maintaining high performance; we can just write code which implements old interface functions by calling new functions. And there are many tools available to ensure that a library is compatible with old interfaces.
Users would still be able to write their own protocol implementations. Indeed, there's no reason to not make the client library open source. Users would simply need to ensure that they were always up to date; downloading the client library on the fly makes that easier, but is not required.
Backwards-compatible, extensible wire protocols are frequently the appropriate choice; but when high performance is needed, different techniques are required. By downloading the client library for the wire protocol on the fly, we can have high performance and extensibility, without compromises.