Giter Club home page Giter Club logo

bscopy's Introduction

BSCopy

BScopy is a set of scripts used to manipulate data in battlescribe/new recruit .cat files.

There are a few main cases:

  • Creating weapon or unit entries (selection entries) from raw text
    • From text pasted into a script
    • From an epub
    • From a custom errata format (that's multiple blocks of pasted text)
  • Coping changes from one .cat file to another

Parse raw text into the proper xml format.

Most of the code for this is set up specifically for Horus Heresy 2.0

There are two scripts to run. Both share the following lines of code which are essentially configuration settings:

page_number = "23"
publication_id = "89c5-118c-61fb-e6d8"
raw_text = """ 
INSERT TEXT HERE
"""
output_file = "unit_output.xml"
  • output_file: The name of the file where the generated XML will be appended. I recommend leaving this as the default.
  • page_number: The page number from the publication source of the data.
  • publication_id: The battlescribe id of the publication entry in the game's dataset.
  • raw_text: a multiline string with specific formatting requirements for each script. See below.

After running each script, you will want to manually copy-paste the generated entries into the end of the shared selection entry list in the XML file. Make a minor change in your data editor to re-save the file and clean up the spacing/formatting.

You will need python3 installed, but no other dependencies. Run each script in the command line as below, or with an IDE such as PyCharm

python3 text_to_weapons.py

This script takes a list of weapons, and creates a selection entry for each. It uses the following format:

raw_text = """
Weapon name <range> <strength> <ap> <type and list of special rules>
Examples:
Three Word Name 12-96“ 7 5 Heavy 5, Barrage, Blast (3”), Pinning, Rending (5+)
Regular Name 24“ 5 4 Rapid Fire, Ignores Cover
Regular Name 24“ 5 4 Rapid Fire 2, Ignores Cover
shortname - +3 - Melee, Overload (6+), Sudden Strike (3)
"""

When running it, I found it easiest to paste all the weapon entries from a page in, a block at a time, and then run the script to add to the output, changed the page number, and then proceeded to the next block, pasting the output after doing all the entries in a publication.

If the message There were one or more errors, please validate the above prints, then something went wrong, likely a special rule was not found. Ensure the special rule is in the game's files.

This script takes a unit entry in the following format, and a couple additional options:

from settings import fast_attack_force_org

force_org = fast_attack_force_org  # whatever force_org slot you need, just ensure it's imported properly.

base_points = 160  # points from the unit name

access_points = ""  # Not adding automatic handling for this.
raw_text = """
UNIT NAME IN ALL CAPS (will be converted to titlecase)
Model Name 7 4 4 4 4 2 4 2 7 4+
Different Model Name 7 4 4 4 4 2 4 3 8 4+

Unit Composition
● 4 Model Name
● 1 Different Model Name
Unit Type
● Model Name: Infantry (Subtype)
● Different Model Name: Infantry
(Subtype, Character)
Wargear
● Wargear List
● More wargear
● Wargear Carbine

Special Rules
● Special rules
● Really long special rule name
that got split on multiple lines

Options:
● A UNIT NAME IN ALL CAPS may include:
- Up to 5 additional Model Name.....................................................+25 points per model
● One Model Name may take a:
- Wargear (any shared selection entry) ...........................................................................................................................+5 points
● The unit may be equipped with any of the following:
- Wargear ....................................................................................................................+30 points
- Other Wargear .................................................................................................................+30 points
● Any model in the unit may exchange their Wargear Carbine for one of the following:
- Wargear A and Wargear B ...................................................................................+5 points
- Common Weapon Name .................................................................................................................+5 points
● The Other Model Name may take one of the following:
- Taser GOAT..........................................................................................................................+5 points
"""

The above example is for a non-vehicle unit, but vehicles also work.

Copy-pasting from PDFs doesn't generally want to read all the text from the page at once, so instead you should copy in chunks. Generally the chunks are Name, statlines, and everything else. The base points, force organization slot, and access points for vehicles need to be added to the respective variables.

The script looks for the named sections, and then splits those options by the ● character into entries it can search. The Special rules section can be omitted and the script will compensate, but leaving out any other section will cause issues. Because the script splits on the ● character, long special rule names, wargear, or unit subtypes that drop to the next line are tolerated.

The points per model defined in the options list (looks for may include) are used in conjunction with the unit cost to set the points for any non-"may take extra" model. In the above example, it will be set to 60.

Options lists are parsed by the ● character per list, and the - character per option on the list, with any number of . characters separating the points and the option. They look for shared selection entries from any .cat or .gst) and group them into Selection Entry Groups, and have the following behavior:

  • Per-model points are read, as described above. Those lines don't get checked for wargear
  • Option titles containing and/or are assumed to have max 2 entries instead of max 1.
  • Option titles containing Any model or the model name are assumed to be model-specific, and will be added to the selection entry of each model instead of the unit.
    • Any entries starting with One ignore this rule, and are placed on the unit.
  • Option titles containing echange will remove whatever wargear is in the title from that model's default wargear, and add it to the selection entry group for this option list for this model instead.
    • It will be the first link or selection entry in that group and will set defaultAmount to 1 or 2.
      • Note that if it is set to 2, you will likely need to manually change it to 1 and add the other and/or option.
    • If there are multiple option titles that could exchange the wargear item, you will have the merge them by hand. This script can only do so much! But it will remind you in the error output.
  • Options lines that end in each (other than the model lines) will need to have a 'multiply by number of models' modifier set manually. Again, the script you will remind you.

As with the weapon script, you can then copy this into the Shared Selection Entry section in your file.

Parse raw text directly from a document

Unlike the other scripts, I can't rely on standard libraries to read epubs (Well, I could, but I don't want too), and beautiful soup makes reading html from the epub much easier. And PDF is even more difficult.

Epub

Install dependencies with pip install -r epub_requirements.txt More documentation forthcoming, but essentially, you can modify settings in read_from_epub.py and run it.

PDF

If an epub is available, there's much more layout data we can easily parse in an epub. Reading from PDF is a WIP.

Install dependencies with pip install -r pdf_requirements.txt. The single requirement is a wrapper for poppler, which has to be installed via Conda.

Copy changes from one file to another

The copying function of BSCopy was set up originally, to copy the shared selection entry links from a template file to the 18 space marine legions of horus heresy, before all the legion-specific units (and rite of war changes?) were added. It likely needs some tweaks now to ensure it does not overwrite any changes and as such I WOULD NOT RECOMMEND USING IT WITHOUT TESTING

It assumes that the only changes you want to copy are selection entry links. This is generally the case when there is one library cat file and multiple files using entries from each of them (with visibility or category changes only).

For instructions on how to use the copying functionality as of when we synced all the legion units, please refer to changes_for_hh_files.md

This is awesome, I'd love to use it for game name here

Cool! You'll probably need to manually edit a lot of the code, to mirror the format of whatever system your coming from. Reach out to @nstephenh in the BSData Discord: Chat on Discord

Other Scripts:

If you've forked a repo, then you need to change the IDs for the gst and each cat file. This script can be run to notify you if any of the cat files ID's need to be changed and updates any catalogue links referencing the old IDs.

If you've forked a repo, and you are trying to pull changes from the upstream repo into your local repo, you will likely have some conflicts. This script's goal is to make conflicts easier to spot as you can keep your branch's revision numbers identical to the source repo.

As a test runner for a game system

Install selenium and webdriver-manager Running system/game/tests/test_game.py will install chromedriver and load new-recruit

bscopy's People

Contributors

nstephenh avatar

Watchers

 avatar

Forkers

the4d6

bscopy's Issues

Duplicate Finder making links to links

Duplicate finder is likely making links to links when replacing nodes.
Instead, update all links pointing to old node with links pointing to new node.

Duplicate finder

Iterate over all objects, look for rules with the same name, profiles with the same stats, etc.
Stretch goal:
If one profile is on a collective object and the original isn't , link them.

Nodes ending up with duplicate children

Listing children of the legion node:
None (constraints None in 2022 - Horus Heresy.gst)
None (constraints None in 2022 - Horus Heresy.gst)
None (selectionEntries None in 2022 - Horus Heresy.gst)
None (selectionEntries None in 2022 - Horus Heresy.gst)

Node tracking

Make a node object that references the tree it's in, the original file, and if that tree needs to be written to disk?

Pull asterisks for unit

If a line at the end of a unit ends in * or ^ then we should pull that and store it somewhere on the raw unit.

Finish create from unit

  • Pintle-mounted needs a set name
  • Handling for "(with co-axial)" (or just make one manually)
  • Weapon A and Weapon B

Handle 3 units on a page

After fixing the "number of units" bug #17, generalize the logic to handle 3+ units on a page (don't think we'll ever see more than 3).

Unit import: be able to create collective entries

Sometimes, collective entries are the way to go. I'm still not 100% when, but if I can figure that out programatically, and automating all the 1 in x constrictions on it, that'd be awesome.

Could also consider a "turn this unit that was non-collective into collective" script.

Get special rules from a units raw text

First get the equipment and special rule names listed in the units description. Then iterate through the lines below and break when we find a special rule we've referenced already.

This might not catch warlord traits in heresy.

We will also want to definitely filter out (see below), possibly add a special check to make sure we find it.

Suggest or auto-add imports

Currently scripts will not import anything. We can add imports ourselves but if a import is unexpected, would be nice to know where it s without searching.

Auto-errata

Auto-errata script and file format.
JSON file with names of things to update

Consider adding last known hash of entry to update

Want an array id changes with target to update and line to do so. Group by page of publication?

Goals of the auto-errata format:

  • Allow changes to be applied, and if a changed is removed from a future errata document, easily remove those changes.
    • Why can't you just do this with a git diff?
      • Because it's hard to tell what's a small data change and what's a conscious errata choice.

Limitations:

  • Not useful when you don't have a base version to compare to.

Specification Draft:

{
  "Document Name": {
    "version": "number",
    "bs_pub_id": "UUID",
    "pages": [
      {
        "page": "number",
        "change list": [
          {
            "Change specification": "<how the change happens somehow>",
            "affected nodes": [
              {
                "ID": "BS ID array, possibly auto-populated?"
              }
            ]
          }
        ]
      }
    ]
  }
}
}

BSCopy removed Retinue (X) entry links from HH files

For some reason the following two entry links were removed here:
https://github.com/BSData/horus-heresy/pull/2550/files
<entryLink id="b856-09d3-d6e5-88cb" name="Retinue (Caster/Speaker)" hidden="true" collective="false" import="true" targetId="e6b4-bd3b-1d71-963c" type="selectionEntryGroup"> from Space Wolves
and
<entryLink id="ac12-0513-0f1c-a95f" name=" Retinue (Stormseer)" hidden="false" collective="false" import="true" targetId="0dfa-aefe-956c-6178" type="selectionEntryGroup"> from White Scars

Fix Note continuation lines also appending as stats

{ # Correct behavior
 "Note": "The crew of a Dreadquake Mortar with an Ogre Loader has +2 Wounds and +2 Attacks, the Weapon Skill, Strength and Initiative of which are shown above."
              }
            },
            { # Incorrect behavior
              "Name": "the",
              "Stats": {
                "M": "Weapon",
                "WS": "Skill,",
                "BS": "Strength",
                "S": "and",
                "T": "Initiative",
                "W": "of",
                "I": "which",
                "A": "are",
                "Ld": "shown",
                "Points": "above. Troop Type:              War machine"
              }

Do not replace list

Add a "Do not replace entries from the following publications" list to ensure entries from ingenium don't replace entries from panoptica.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.