However, among those challenges to this new approach would be safely passing richly-structured data throughout the API border in a manner that’s memory-safe and operates nicely with Rust’s ownership system. Serializing our data structures implemented our first versions and in a few instances returning a struct by pointer. The process for returning, say, a bookmark in the consumer ’s synced data looked like this:
The biggest and most obvious benefit is that we get one canonical code base, written in a safe and statically typed language, deployable on each stage. Every business logic change that is upstream becomes accessible to all our products using a version bump.
We quickly realized there was likely a faster and better way that could be safer than our solution.
How we solved the FFI challenge safely
Firefox iOS has begun to oxidize by substituting their password syncing motor using the one we constructed in Rust. The plan is to do the same on Firefox Desktop also.
Solving for one too many targets
One thing I need to mention: As we ship both the manufacturer and consumer of those binary flows as a device, we’ve got the freedom to change our information exchange format transparently without affecting our Android and iOS customers at all.
The post Crossing the Rust FFI frontier with Protocol Buffers appeared first on Mozilla Hacks – the Internet programmer blog.
A cross-platform approach with Rust
Returning Bookmark info from Rust into Kotlin using Protocol Buffers two (simplified)
These attributes are shipped on iOS browsers, Android and Firefox Desktop. They will be available in our new products such as our Android browser, our manager Lockbox, and Firefox .
There are a few drawbacks to this strategy: it’s more work to convert our inner types to the created protobuf structs –e.g a
url::Url needs to be converted to a String original — whereas back when we had been using serde-json serialization, any struct implementing
serde::Serialize was a line from being sent within the FFI barrier. It adds yet another step although it was easy to integrate.
Protoc (the Protocol Buffers code generator) can emit code in over 20 languages! On the Rust facet, we utilize the prost cage that outputs really clean-looking structs by leveraging Rust derive macros.
Returning Bookmark info from Rust to Kotlin using JSON (simplified)
- Performance: JSON serializing and de-serializing is notoriously slow, because functionality wasn’t a primary design goal for its structure. On the Java coating, an excess string copy happens on top of this Java strings are UTF-16-ish and because Rust strings are UTF-8. At scale, it can introduce substantial overhead.
- Complexity and safety: every information structure is manually parsed and deserialized from JSON strings. A data structure field modification on the Rust side must be mirrored on the Kotlin facet, or an exception will most likely occur.
- Even worse, in some instances we were not returning JSON strings but C-shaped Rust structs by pointer: forget to update the Construction Kotlin subclass or the Objective-C struct and you get a severe memory corruption onto your hands.
Looking forward, there’s probably a high-level system which may be used to exchange data within the FFI according to Rust macros. There’s been talk about using FlatBuffers to squeeze even more functionality out. In our case Protobufs provided the right trade-off between ease-of-use, functionality, and comparative safety.
The –relative– safety comes in the automatic creation of data structures in the languages we all care about. There is just one source of fact –that the .proto schema file–where all of our data classes are generated at build time, both on the consumer and Rust side.
And needless to say, on top of this, Protocol Buffers are than JSON.
Considering the magnitude of our team, we immediately realized that our current approach to transport products would not scale across more goods and could lead to quality issues like bugs or irregular feature-completeness across platform-specific implementations. We decided to plan for the future. Mozilla is now making a bet on the new Rust programming language, so it was normal for us to follow suit, as you might be aware.
Luckily, you will find many info serialization formats out there which aim to be fast. Those with a schema language may data arrangements !
After some exploration, we ended up settling on Protocol Buffers version 2.
Our new approach is as follows: We shall build cross-platforms parts, implementing our core business logic utilizing Rust and wrap it in a thin platform-native coating, for example Kotlin for Android and Swift to get iOS.
If you are interested in helping us build the future of Firefox Sync and much more, or simply following our progress, head to the application services Github repository.
What’s the issue with this approach?
A look ahead