Puppet Editor Syntax
Syntax highlighting files for editors (VSCode, Atom, SublimeText, TextMate, etc.) for the Puppet Language
Information
What is this for
There were many locations with different formats of the Textmate style syntax highlighting; Each editor has it's own format, or there were many plugins/packages/extensions which added the syntax highlighting.
This repository is hoping to house syntax highlighting files for most editors. It uses the Textmate XML Property List file as the master template and then generates other formats based on that.
How to use the syntax files
Copy the right file type for your editor. Currently the XML Plist file is in the /syntaxes
directory and JSON, YAML, CSON and atom flavoured CSON files are in the /generated-syntaxes
directory.
How does the syntax parser work
Unfortunately it's too complex to describe in a README, but in essence, the parser is a great big regex engine. It goes through a list of patterns looking for match. When it finds a match, it assigns a scope to the text, and then starts again with the next bit of text.
At the end of parsing, the entire document is split out into tokens which contains the text, and the scope assigned to that text.
Theme editors can then assign colours or formatting to scopes, for example;
After parsing a document, a token with text $myvar
has a scope variable.other.puppet
assigned. The theme in the text editor says that the scope variable.other.puppet
should have red text. So now in your editor $myvar
will appear in red!
Helpful links
Textmate Manual - Language Grammars
Writing a TextMate Grammar: Some Lessons Learned
What scopes should I use
While it is up to the theme to determine what scopes each syntax should use, I found Sublime Text had good documentation on when and what scopes to use.
https://www.sublimetext.com/docs/3/scope_naming.html
Developing and Contributing
Setup
This project is cross platform, and can be developed using Mac, Linux or Windows.
-
Install NodeJS (This project tests on NodeJS v9)
-
Install node modules using
npm install
Note on Windows you may need to install the Windows build tools using;
npm install --global --production windows-build-tools
Using Gitpod
If you're using gitpod, you can create a workspace using:
https://gitpod.io/#https://github.com/<Github Username>/puppet-editor-syntax/tree/<Git Branch>
To just browse master:
https://gitpod.io/#https://github.com/puppetlabs/puppet-editor-syntax
Running tests
This project makes use of mocha and expect.js. The tests are located in the /tests
directory.
To run tests use;
npm run test
Converting to other formats
This project can automatically convert an XML Plist file into other formats.
To run the converter use;
npm run convert
Debugging tools
This project makes use of Visual Studio Code's textmate parser. In order to make development easier, the parser can be put into a debug mode where it shows each and every regex match applied and how an entire file is processed.
To run the inspector use;
npm run inspect <mainGrammarPath> [<additionalGrammarPath1> ...] <filePath>
For example,
To see how a file called C:\Temp\testfile.pp
is parsed;
> npm run inspect syntaxes/puppet.tmLanguage C:\Temp\testfile.pp
LOADING GRAMMAR: .\syntaxes\puppet.tmLanguage
===========================================
TOKENIZING LINE 1: |Array[String] $testvar|
@@scanNext: |Array[String] $testvar\n|
scanning for
- 3: ^((#).*$\n?)
- 6: (#).*$\n?
- 9: \b(absent|directory|false|file|present|running|stopped|true)\b(?!.*{)
- 10: ^\s*/\*
- 11: ^\s*(node|class)\s+((?:[-_A-Za-z0-9"\'.]+::)*[-_A-Za-z0-9"\'.]+)\s*
- 60: ^\s*(plan)\s+((?:[a-z][a-z0-9_]+::)*[a-z][a-z0-9_]+)\s*
- 75: ^\s*(define|function)\s+([a-zA-Z0-9_:]+)\s*(\()
- 90: ^\s*(\w+)\s*{\s*([\'"].+[\'"]):
- 93: ^\s*(\w+)\s*{\s*(\$[a-zA-Z_]+)\s*:
- 96: \b(case|if|else|elsif|unless)(?!::)\b
- 59: (?<![a-zA-Z\$])([A-Z][a-zA-Z]*)(?![a-zA-Z])
- 32: "
- 42: '
- 53: (\[)
- 97: ((\$?)"?[a-zA-Z_\x{7f}-\x{ff}][a-zA-Z0-9_\x{7f}-\x{ff}]*"?):(?=\s+|$)
- 46: (?<!\w|\d)([-+]?)(?i:0x)(?i:[0-9a-f])+(?!\w|\d)
- 47: (?<!\w|\.)([-+]?)(?<!\d)\d+(?i:e(\+|-){0,1}\d+){0,1}(?!\w|\d|\.)
...
Creating Pull Requests
-
Please add tests for any changes
-
Remember to commit the generated syntaxes too. Pull Request CI will fail if it finds that this has not been done.