ACL Wiki - User contributions [en]

Hypergraph Format

2010-11-07T01:49:42Z

Srush:

== JSON ==

[http://www.json.org/ JSON Description]

Pro:
* Implementations in every language (often packaged with language).
* Human readable
* Already used in CDec for forest output

Con:
* Space inefficiency

== Protocol Buffers ==

[http://code.google.com/p/protobuf/ Protocol Buffer Description]

[http://github.com/srush/hypergraph Implementation Sketch]

Pro:
* Conversion to and from JSON ([http://code.google.com/p/protobuf-json/ protobuf-json])
* Very fast to read (particularly in C++ and Java, hopefully soon in python)
* Very space efficient
* Implementations in every language (although requires a separate library)
* Automatically generates typed stubs

Con:
* "It's really easy to get up to some of the data size
limits that are in place to prevent malicious data from having the PB
parser allocate too much memory"
* "You typically have to create a full hypergraph protocol buffer object before you can serialize it, so you either have to use the PB data structures internally in your code or you have to copy your data structure. While doing this copy, you can end up with two copies of the forest in memory, which is bad for memory usage."

== Variation of SLF (Standard Lattice Format) ==

[http://labrosa.ee.columbia.edu/doc/HTKBook21/node257.html SLF Specification]

Pro:
* Blindingly fast.
* Could be implemented to work lazy/streaming.

Con:
* Requires a custom format
* Probably need specialized language bindings.

== Tiburon Format ==

[http://www.isi.edu/licensed-sw/tiburon/ Tiburon Specification]

Hypergraph Format

2010-11-07T01:43:52Z

Srush:

== JSON ==

[http://www.json.org/ JSON Description]

Pro:
* Implementations in every language (often packaged with language).
* Human readable

Con:
* Space inefficiency

== Protocol Buffers ==

[http://code.google.com/p/protobuf/ Protocol Buffer Description]

[http://github.com/srush/hypergraph Implementation Sketch]

Pro:
* Conversion to and from JSON ([http://code.google.com/p/protobuf-json/ protobuf-json])
* Very fast to read (particularly in C++ and Java, hopefully soon in python)
* Very space efficient
* Implementations in every language (although requires a separate library)

Con:
* "It's really easy to get up to some of the data size
limits that are in place to prevent malicious data from having the PB
parser allocate too much memory"
* "You typically have to create a full hypergraph protocol buffer object before you can serialize it, so you either have to use the PB data structures internally in your code or you have to copy your data structure. While doing this copy, you can end up with two copies of the forest in memory, which is bad for memory usage."

== SLF (Standard Lattice Format) ==

[http://labrosa.ee.columbia.edu/doc/HTKBook21/node257.html SLF Specification]

Hypergraph Format

2010-11-07T01:43:20Z

Srush:

== JSON ==

[http://www.json.org/ JSON Description]

Pro:
* Implementations in every language (often packaged with language).
* Human readable

Con:
* Space inefficiency

== Protocol Buffers ==

[http://code.google.com/p/protobuf/ Protocol Buffer Description]

[http://github.com/srush/hypergraph Implementation Sketch]

Pro:
* Conversion to and from JSON ([http://code.google.com/p/protobuf-json/ protobuf-json])
* Very fast to read (particularly in C++ and Java, hopefully soon in python)
* Very space efficient
* Implementations in every language (although requires a separate library)

Con:
* "It's really easy to get up to some of the data size
limits that are in place to prevent malicious data from having the PB
parser allocate too much memory"
* "You typically have to create a full hypergraph protocol buffer object before you can serialize it, so you either have to use the PB data structures
internally in your code or you have to copy your data structure. While
doing this copy, you can end up with two copies of the forest in
memory, which is bad for memory usage."

== SLF (Standard Lattice Format) ==

[http://labrosa.ee.columbia.edu/doc/HTKBook21/node257.html SLF Specification]

Hypergraph Format

2010-11-07T01:37:48Z

Srush:

== JSON ==

[http://www.json.org/ JSON Description]

== Protocol Buffers ==

[http://code.google.com/p/protobuf/ Protocol Buffer Description]

[http://github.com/srush/hypergraph Implementation Sketch]

== SLF (Standard Lattice Format) ==

[http://labrosa.ee.columbia.edu/doc/HTKBook21/node257.html SLF Specification]

Hypergraph Format

2010-11-07T01:34:29Z

Srush:

* JSON

* Protocol Buffers

[http://github.com/srush/hypergraph Implementation Sketch]

* SLF

Hypergraph Format

2010-11-07T01:34:12Z

Srush:

* JSON

* Protocol Buffers

[http://github.com/srush/hypergraph Implementation Sketch]

* SLF

Hypergraph Format

2010-11-07T01:33:24Z

Srush:

* JSON

* Protocol Buffers

[[Implementation Sketch | http://github.com/srush/hypergraph]]

* SLF

Hypergraph Format

2010-11-07T01:32:54Z

Srush:

* JSON

* Protocol Buffers

Implementation Sketch: [[http://github.com/srush/hypergraph]]

* SLF

Hypergraph Format

2010-11-07T01:32:10Z

Srush: New page: * JSON * Protocol Buffers * SLF

* JSON

* Protocol Buffers

* SLF