Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CS2] Literate CoffeeScript without dependencies #4509

Merged
merged 4 commits into from
Apr 18, 2017
+110 −131
Merged

[CS2] Literate CoffeeScript without dependencies #4509

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
b637592
Reimplement `invertLiterate` without any dependency, tracking indenta…
GeoffreyBooth Apr 16, 2017
44049ae
Drop Literate CoffeeScript’s support for executable code blocks insid…
GeoffreyBooth Apr 17, 2017
6d488f1
Update Literate CoffeeScript docs to reflect current supported syntax
GeoffreyBooth Apr 17, 2017
24c6c21
Merge branch '2' into litcoffee
GeoffreyBooth Apr 18, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 46 additions & 25 deletions lib/coffeescript/helpers.js
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 2 additions & 3 deletions package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,8 @@
"google-closure-compiler-js": "^20170409.0.0",
"highlight.js": "~9.10.0",
"jison": ">=0.4.17",
"markdown-it": "^8.3.1",
"underscore": "~1.8.3"
},
"dependencies": {
"markdown-it": "^8.3.1"
}
"dependencies": {}
}
58 changes: 45 additions & 13 deletions src/helpers.coffee
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@
# the **Lexer**, **Rewriter**, and the **Nodes**. Merge objects, flatten
# arrays, count characters, that sort of thing.

md = require('markdown-it')()

# Peek at the beginning of a given string to see if it matches a sequence.
exports.starts = (string, literal, start) ->
literal is string.substr start, literal.length
Expand Down Expand Up @@ -69,20 +67,54 @@ exports.some = Array::some ? (fn) ->
return true for e in this when fn e
false

# Simple function for extracting code from Literate CoffeeScript by stripping
# Helper function for extracting code from Literate CoffeeScript by stripping
# out all non-code blocks, producing a string of CoffeeScript code that can
# be compiled “normally.” Uses [MarkdownIt](https://markdown-it.github.io/)
# to tell the difference between Markdown and code blocks.
# be compiled “normally.”
exports.invertLiterate = (code) ->
out = []
md.renderer.rules =
# Delete all other rules, since all we want are the code blocks.
code_block: (tokens, idx, options, env, slf) ->
startLine = tokens[idx].map[0]
lines = tokens[idx].content.split '\n'
for line, i in lines
out[startLine + i] = line
md.render code
blankLine = /^\s*$/
indented = /^[\t ]/
listItemStart = /// ^
(?:\t?|\ {0,3}) # Up to one tab, or up to three spaces, or neither;
(?:
[\*\-\+] | # followed by `*`, `-` or `+`;
[0-9]{1,9}\. # or by an integer up to 9 digits long, followed by a period;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just curious where you got this "up to 9 digits long" rule from? :)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

http://spec.commonmark.org/0.27/#list-items

)
[\ \t] # followed by a space or a tab.
///
listItemFirstCharacter = new RegExp "#{listItemStart.source}+(\\S)"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be written as ///#{listItemStart.source}+(\S)///

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, why is listItemStart matched one or more times (+)?

insideComment = no
insideList = no
indentation = 0
for line in code.split('\n')
if blankLine.test(line)
insideComment = no
out.push line
else if listItemStart.test(line)
insideList = yes
# Get the first non-whitespace character after the bullet or period, to
# determine our new base indentation level.
indentation = line.indexOf line.match(listItemFirstCharacter)[1]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't this crash if the listItemFirstCharacter doesn't match? I'm thinking that a line only containing a "listItemStart" but nothing after it could cause such a crash?

-

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we are looking for something more like:

match = listItemStart.exec line
if match
  indentation = line[match[0].length..].search /\S/
  # TODO: `.search` returns `-1` if nothing could be found

out.push "# #{line}"
else if indented.test(line)
if insideComment
out.push "# #{line}"
else if insideList
# Are we in a list, and indented relative to the indentation of the list?
# If so, treat as code.
if indented.test(line.substring(indentation))
out.push line.substring(indentation)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In CoffeeScript I'd say it's more common to use slicing syntax. line[indentation..]. Saves a trip to MDN.

# Otherwise this is part of a list item paragraph.
else
out.push "# #{line}"
else
insideComment = no
out.push line
else
insideComment = yes
insideList = no
indentation = 0
out.push "# #{line}"
out.join '\n'

# Merge two jison-style location data objects together.
Expand Down
21 changes: 16 additions & 5 deletions test/literate.litcoffee
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,11 @@

comment comment

testsCount = 0 # Track the number of tests run in this file, to make sure they all run

test "basic literate CoffeeScript parsing", ->
ok yes
testsCount++

now with a...

Expand All @@ -16,6 +19,7 @@ now with a...
... nested block.

ok yes
testsCount++

Code must be separated from text by a blank line.

Expand All @@ -25,6 +29,7 @@ The next line is part of the text and will not be executed.
fail()

ok yes
testsCount++

Code in `backticks is not parsed` and...

Expand All @@ -38,6 +43,7 @@ Code in `backticks is not parsed` and...
###

ok yes
testsCount++

Regular [Markdown](http://example.com/markdown) features, like links
and unordered lists, are fine:
Expand All @@ -50,11 +56,6 @@ and unordered lists, are fine:

* List

Tabs work too:

test "tabbed code", ->
ok yes

---

# keep track of whether code blocks are executed or not
Expand All @@ -69,13 +70,15 @@ if true

test "should ignore code blocks inside HTML", ->
eq executed, false
testsCount++

---

* A list item with a code block:

test "basic literate CoffeeScript parsing", ->
ok yes
testsCount++

---

Expand Down Expand Up @@ -134,6 +137,7 @@ and test...

test "should recognize indented code blocks in lists with empty line as separator", ->
ok executed
testsCount++

---

Expand All @@ -146,6 +150,7 @@ and test...

test "should ignore indented code in escaped list like number", ->
eq executed, no
testsCount++

one last test!

Expand All @@ -155,3 +160,9 @@ one last test!
and bar!
'''
eq quote, 'foo\n and bar!'
testsCount++

and finally, how did we do?

test "all spaced literate CoffeeScript tests executed", ->
eq testsCount, 9
27 changes: 19 additions & 8 deletions test/literate_tabbed.litcoffee
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,11 @@

comment comment

testsCount = 0 # Track the number of tests run in this file, to make sure they all run

test "basic literate CoffeeScript parsing", ->
ok yes
testsCount++

now with a...

Expand All @@ -16,6 +19,7 @@ now with a...
... nested block.

ok yes
testsCount++

Code must be separated from text by a blank line.

Expand All @@ -25,6 +29,7 @@ The next line is part of the text and will not be executed.
fail()

ok yes
testsCount++

Code in `backticks is not parsed` and...

Expand All @@ -38,6 +43,7 @@ Code in `backticks is not parsed` and...
###

ok yes
testsCount++

Regular [Markdown](http://example.com/markdown) features, like links
and unordered lists, are fine:
Expand All @@ -50,11 +56,6 @@ and unordered lists, are fine:

* List

Spaces work too:

test "spaced code", ->
ok yes

---

# keep track of whether code blocks are executed or not
Expand All @@ -69,13 +70,15 @@ if true

test "should ignore code blocks inside HTML", ->
eq executed, false
testsCount++

---

* A list item with a code block:

test "basic literate CoffeeScript parsing", ->
ok yes
test "basic literate CoffeeScript parsing", -> # Needs to be indented with spaces up to the indentation level of the list
ok yes
testsCount++

---

Expand Down Expand Up @@ -128,12 +131,13 @@ This is [an example][id] reference-style link.

1986. What a great season.

executed = yes
executed = yes # Needs to be indented with spaces up to the indentation level of the list

and test...

test "should recognize indented code blocks in lists with empty line as separator", ->
ok executed
testsCount++

---

Expand All @@ -146,6 +150,7 @@ and test...

test "should ignore indented code in escaped list like number", ->
eq executed, no
testsCount++

one last test!

Expand All @@ -155,3 +160,9 @@ one last test!
and bar!
'''
eq quote, 'foo\n\t\tand bar!'
testsCount++

and finally, how did we do?

test "all tabbed literate CoffeeScript tests executed", ->
eq testsCount, 9