<p>Hi everyone,</p>
<p>I started a project a while ago on (<a href="https://github.com/iwalz/tdoc" rel="nofollow">github</a>) and I felt kinda stuck at one stage, so I decided to fire of a call for help here to proceed with this project - cause I feel this would be helpful for more people than just me.</p>
<p>This project has no documentation for now by intention, cause it's in a very early development stage and I didn't want to raise any attention before it's usable. So here a small description what it's supposed to be.</p>
<p><strong>About tdoc</strong></p>
<p>tdoc is supposed to generate SVG pictures from text - the source for the big picture are actually plain SVG files (like the official iconset from AWS, Azure or whatever). If you want to generate an AWS documentation, it should be as easy as that:</p>
<pre><code>sns foo -> lambda bar
</code></pre>
<p>This will take the picture "sns.svg", labels it with foo and draws a relation to the bar labeled "lambda.svg".
Fair enough, the problem is, it should be very flexible to also support input like this:</p>
<pre><code>dc datacenter1 {
ec2 foo {
docker my_container1
}
}
dc datacenter2 {
ec2 bar {
docker my_container2
}
}
</code></pre>
<p>So therefor I decided to use YACC. </p>
<p><strong>Current state of the project</strong></p>
<p>tdoc is able to render something like this correct:</p>
<pre><code>dc datacenter1 {
ec2 foo
lambda bar
sns blubb
}
</code></pre>
<p><a href="http://i.imgur.com/khATEYe.png" rel="nofollow">Imgur</a>
The biggest issue I'm facing currently, is the grammar for the multi-nested components. Rendering the images nested on many layers does work - but the parsing of the initial input is still an issue. </p>
<p><a href="http://i.imgur.com/lIklUIp.png" rel="nofollow">Imgur</a></p>
<pre><code>cloud foo {
vpc bar
client blubb
APIGateway baz
CloudWatch_alarm quo
CloudSearch blubbb
DynamoDB_item bazz
MachineLearning test {
EC2_instance bar1
EC2_instance bar2
EC2_instance bar3
EC2_instances bar4
EC2_instances bar5
EC2_instances bar6
OpsWorks_instances bar7
RDS_RDSDBinstance bar8
RDS_OracleDBinstancealternate bar9
}
}
</code></pre>
<p>The smallest samle that doesn't work:
<a href="http://i.imgur.com/hcU6uOL.png" rel="nofollow">Imgur</a></p>
<pre><code>cloud foo {
vpc bar {
EC2_Instance blubb
}
vpc baz {
EC2_Instance quo
}
}
</code></pre>
<p>The source for this problem is actually the design of the grammar - but since I've absolutely no experience with YACC (beside what I've learned so far while working on tdoc), I'd really like to get some help and a review of the <a href="https://github.com/iwalz/tdoc/blob/master/parser/tdoc.y" rel="nofollow">grammar</a>.
The relations are not yet build into the grammar, since the nesting is causing me so much pain.</p>
<p><strong>Try it yourself</strong></p>
<p>After you go-get'ed tdoc, you can try it yourself with just putting any random svg in a folder (like test.svg) and run it with:
tdoc -s /home/foo/svg input.tdoc</p>
<p>And input.tdoc would look like:
test foo</p>
<p>Any feedback, review, support, PR, chat - whatever would be welcome!</p>
<hr/>**评论：**<br/><br/>sin2pifx: <pre><p>You might want to take a look at <a href="http://www.graphviz.org/" rel="nofollow">graphviz</a>.</p></pre>ducky_cloud: <pre><p>I already made a lot of researches in this area - sth like this does just not exist. The problem is mostly that tools like graphviz or plantuml are not very good when it comes to drawing custom icons. The main purpose was to draw a picture like you'd scetch sth on the whiteboard, using official iconset.</p></pre>sin2pifx: <pre><p>My hunch is that "depth" is not correct. YACC (or probably bison) executes the code blocks after the rule has been recognized. declaration is left recursive, so it might be incrementing depth too late. It usually is not a good idea to shadow the state of a parser in a global variable. The cleanest solution is build the parse tree and then traverse the tree to perform the actions.</p></pre>ducky_cloud: <pre><p>Do you have any ressource to look it up or a hint for an implementation? The idea of a parser tree is not new to me (tried it), but I faced several issues because of the limited number of non-terminals and the highly recursive grammar ... </p>
<p>Depth is indeed my biggest problem - there's another branch (root_rewrite) which uses x and y (which depth and the number of the component in depth to add the component to). And as you said - under specific circumstances, this state is just not valid anymore ... </p></pre>sin2pifx: <pre><p>I'll try to have a look tomorrow evening.</p></pre>sin2pifx: <pre><p>RemindMe! 20 hours</p></pre>9nut: <pre><p>based on a cursory review of <code>tdoc.y</code>, i'm not sure why <code>root</code> is an array/slice.</p>
<p>a generic root should be <strong>an element</strong> that contain all other top level elements (e.g. list) and all other elements (and blocks) should be descended from those (i.e. abstract syntax tree). evaluating the tree in order, depth-first, should produce the complete svg string.</p>
<p>it's hard to tell from the grammar what is a definition and what is an invocation. i would consider making those more clear. for example, with something like:</p>
<pre><code>bar : vpc {
EC2_Instance %1
}
baz : vpc {
EC2_Instance %1
}
foo : cloud {
bar %1 '->' baz %2
}
$main : {
foo "blubb" "quo"
}
</code></pre>
<p>assuming i recorded every definition in the registry, <code>registry["foo"]</code> will contain the root element for definition of <code>foo</code> (i.e. a <code>cloud</code>), etc. assuming i have a "runner" that takes a definition and a lookup table (i.e. registry), i should be able to do <code>runner.Eval(registry.Get("$main"), registry)</code> to evaluate the top level program.</p>
<p>here's a good reference for a C version of something similar, the UNIX and Plan 9 <a href="https://github.com/0intro/plan9/blob/7524062cfa4689019a4ed6fc22500ec209522ef0/sys/src/cmd/pic/picy.y" rel="nofollow">pic troff preprocessor</a> grammar.</p></pre>ducky_cloud: <pre><p>Root contains only a reference to other components - it's just a kind of help to know on which component I have to append another component.</p>
<pre><code>vpc foo {
EC2_Instance bar
}
</code></pre>
<p>The component foo will be used to add another component bar and creates a tree in this case. The root_rewrite branch follows the same idea, but multi-dimensional. The the end I have the program component, which is itself 1 component and is the first node for the abstract syntax tree. </p>
<p>Thanks for the reference and your thoughts - will have a look!</p></pre>sin2pifx: <pre><p>I can't get it to compile. When I run make, it says: go tool: no such tool "yacc", so I've got no idea what to do.</p>
<p>Instead, I'll summarize what you might want to do. This is your basic grammar (I noticed missing semicolons BTW):</p>
<pre><code>program: statement_list;
statement_list: statement | statement_list statement;
statement: declaration | relation_assignment;
relation_assignment:
TEXT RELATION TEXT |
TEXT RELATION declaration |
declaration RELATION TEXT |
relation_assignment RELATION declaration |
declaration RELATION declaration;
declaration:
COMPONENT IDENTIFIER |
COMPONENT IDENTIFIER ALIAS TEXT |
declaration SCOPEIN |
SCOPEOUT;
</code></pre>
<p>There does not seem to be a relation between {, SCOPEIN, and }, SCOPEOUT. That's weird. Normally, when you parse, that's the most important anchor. I think your grammar would accept }}}}}.</p>
<p>From what I get from your examples, you actually want something like this (but note that I don't know what TEXT, RELATION, COMPONENT and IDENTIFIER mean):</p>
<pre><code>document: declaration_list { document = $1; }.
declaration_list:{ declaration_list = nil; } |
declaration declaration_list { declaration_list = concatenate($1, $2); };
declaration:
COMPONENT IDENTIFIER { declaration = MakeTerminalNode($1, $2); };
COMPONENT IDENTIFIER SCOPEIN declaration_list SCOPEOUT { declaration = MakeEmbeddingNode($1, $2, $4); };
</code></pre>
<p>That should give you your tree, which you can then process.</p></pre>ducky_cloud: <pre><p>Thanks, I'm gonna change the Makefile.
Go tool yacc has been removed, so goyacc has to be used in your version of go: <a href="https://godoc.org/golang.org/x/tools/cmd/goyacc" rel="nofollow">https://godoc.org/golang.org/x/tools/cmd/goyacc</a></p>
<p>(Just <em>go get golang.org/x/tools/cmd/goyacc</em> and change go tool yacc to goyacc - my local version is a bit messed up compared to github due to all my experiments ...)</p>
<p>You can use sth like</p>
<pre><code>EC2_Instance "This is an Instance" as ec2
</code></pre>
<p>In this case, EC2_Instance is the component, "This is an Instance" is TEXT and ec2 is the IDENTIFIER. RELATION is if you do that</p>
<pre><code>EC2_Instance instance1 -> EC2_Instance instance2
</code></pre>
<p>Let's see if I got it right.</p>
<p>concatenate will return a new Node with $1 <strong>and</strong> $2</p>
<p>MakeTerminalNode just creates a new <strong>component</strong></p>
<p>MakeEmbeddingNode creates a new component and adds it as a child - to what exactly? From where do I know to which component I should add the new Node? </p></pre>sin2pifx: <pre><p>The "EmbeddingNode" would be a container for other components or "embedding nodes". Suppose you make the struct as simple as this:</p>
<pre><code>struct Node {
component string
relation string
children []Node*
}
</code></pre>
<p>you would have all information from the file. The parser then builds it the tree bottom-up. When it recognizes a simple component, it returns a struct where children is nil; when it recognizes COMPONENT IDENTIFIER { ... } it returns a struct with a list of Nodes in children. Wouldn't that take care of the basic nesting problem?</p>
<p>I have to say I don't quite understand what you want to achieve with relation and text, but you can either shoehorn them in the same struct, or have an interface "ParseTreeNode" or something like that and create a list of those as children.</p></pre>ducky_cloud: <pre><p>Relation and text is just an Implementation detail. If you want to display a different text than the ID, you can use text. RELATION draws the lines between two related COMPONENTs and changes the placement.</p>
<p>I don't see for now how the recursion works, but I'll investigate some time tomorrow and see if I can make it work. If it's really that "simple", I'd bite my ass xD That would indeed solve my nesting problem. Thanks!!!</p></pre>ducky_cloud: <pre><p>I just pushed a change. If you want to compile and test, just use</p>
<pre><code>make yacc
go run main.go foo.tdoc
</code></pre></pre>
