Recently, I was having a somewhat heated discussion with a friend about the Java I/O library (specificially
java.io.*). His position was that the library is unnecessarily cluttered and verbose, and that I/O in C is much simpler and more productive. Whilst I agreed with some of that, I also argued that the Java I/O library is powerful and more flexible than C. Here are some of the main points we covered:
- Abstracting stream encodings away from stream processing. The ability for objects in Java to delegate to an
OutputStreamprovides a very nice way to decouple the encoding of information from the processing of it. An interesting example in the Whiley compiler is that of name mangling. That is, encoding the Whiley type of a function into its name. To generate the mangling, I have an
OutputStreamimplementation which takes a Whiley type and serialises it into binary data. Then, a second implementation of
OutputStreamtakes an arbitrary stream of binary data and encodes it into (roughly speaking) 7-bit ASCII — in other words, it turns the serialised data into a form that can safely be used as part of a method name according to the JVM Spec. And, of course, I have the mirror of this for reading the type out of a mangling.
- Stateful encoders/decoders are important. One idea we discussed was that encoding could be handled if: (1) Java had support for lambdas; and (2) classes like E.g.
FileReaderaccepted a decoding (lambda) function. This would cover a large number of use cases, and conceptually simplify the I/O framework. However, we would need our decoding functions to have state to be useful (and I don’t believe that JSR 335 supports this). State is necessary typically in situations where we can — or want — to read chunks larger than necessary. For example, when performing some kind of buffering we read a large chunk and cache it for later. Other examples of stateful stream components include unusual things, such as providing the ability to insert logging into the pipeline.
At this point, things were all making sense to me. Having worked with C for many years (albeit some time ago now), I’m fairly familiar with how I/O is handled in C. In particular, I/O mostly goes through the
FILE* structure (although you can read/write to file descriptors directly if you like). You have to rely on functions like
popen() to create
FILE* instances because we don’t know the internal layout of
FILE* and, hence, cannot construct our own instances. So, my first attack on this structure was to say something like:
“How do you create a FILE* instance from a memory buffer?“
In Java, this is relatively easy since we can create e.g. a
ByteArrayInputStream and pass that to anything accepting an
InputStream. Well, it turns out you can do this in C! I had never heard of it before, but there is a
fmemopen() function for exactly this use case.
Undeterred, I countered with:
“How do you create a FILE* instance which automagically encodes/decodes into a user-defined format (e.g. as per my name mangling example above)?”
Again, this is relatively easy to do in Java by providing your own implementation of e.g.
InputStream. At this point, there was a long pause (in fact, overnight) in the discussion. The next day, my friend comes back and says: “ah, you obviously haven’t heard of the
fopencookie() function then!”. Nope, I hadn’t.
This is an excerpt from the manpage on
The fopencookie() function allows the programmer to create a custom implementation for a standard I/O stream. This implementation can store the stream's data at a location of its own choosing; for example, fopencookie() is used to implement fmemopen(3), which provides a stream interface to data that is stored in a buffer in memory. In order to create a custom stream the programmer must: * Implement four "hook" functions that are used internally by the standard I/O library when performing I/O on the stream. * Define a "cookie" data type, a structure that provides bookkeeping information (e.g., where to store data) used by the aforementioned hook functions. The standard I/O package knows nothing about the contents of this cookie (thus it is typed as void * when passed to fopencookie()), but automatically supplies the cookie as the first argument when calling the hook functions. * Call fopencookie() to open a new stream and associate the cookie and hook functions with that stream. ...
Well, I guess you learn a new thing every day …