Giter Club home page Giter Club logo

tivars_lib_py's Introduction

tivars_lib_py

tivars_lib_py is a Python package for interacting with TI-(e)z80 (82/83/84 series) calculator files, i.e. lists, programs, matrices, appvars, etc.

Much of the functionality of this package has been ported over from tivars_lib_cpp. However, a number of changes have been made to the API to better suit Python's strengths and capabilities as a language (e.g. scripting, dynamic typing).

Installation

The current release version is v0.9.1. All versions require Python 3.10+ to run.

As a Package

Install the tivars package from PyPI using pip:

pip install tivars

Alternatively, you can clone this repository or download a release and extract the tivars directory to include it in your next project. Once downloaded, you can also use pip to install it locally.

As a Submodule

Include this repository in your next project as a submodule using the git submodule command. Then, add the following to any file which imports tivars:

import sys

sys.path.insert(1, 'tivars_lib_py/')

Check out this tool for an example.

Unit Testing

You can run the test suite via __main__.py, or run individual tests found in tests/ with unittest. Tests for optional package extensions (e.g. PIL) will be skipped if the package cannot be found.

Warning

The PyPI distribution does not include the test suite.

How to Use

Creating objects

Var basics

Every var file has two parts: a header and a number of entries, where an entry contains the data for a single variable. Usually, var files contain just one entry; in these cases, there's not much distinction between a var and an entry for the purposes of messing with its data.

Entries

To create an empty entry, instantiate its corresponding type from tivars.types. You can specify additional parameters as you like:

from tivars.models import *
from tivars.types import *

my_program = TIProgram(name="HELLO")

Tip

If you're not sure of an entry's type, you can instantiate a base TIEntry.

Vars and Headers

If you want to create an entire var or just a header, use TIVar or TIHeader instead:

from tivars.var import *

my_var = TIVar()
my_var_for84pce = TIVar(model=TI_84PCE)

my_header = TIHeader()
my_header_with_a_cool_comment = TIHeader(comment="Wow! I'm a comment!")

Reading files

Vars

Vars can be loaded from files or raw bytes:

my_var = TIVar.open("HELLO.8xp")

with open("HELLO.8xp", 'rb') as file:
    my_var.load_var_file(file)
    
    file.seek(0)
    my_var.load_bytes(file.read())

Important

When loading from a file object, make sure the file is opened in binary mode.

Entries

Entries can be loaded from files or raw bytes. When loading from a file, you may specify which entry to load if there are multiple:

# Raises an error if the var has multiple entries
my_program = TIProgram.open("HELLO.8xp")

with open("HELLO.8xp", 'rb') as file:
    # Offset counts the number of entries to skip; defaults to zero
    my_program.load_from_file(file, offset=1)
    
    file.seek(0)
    my_program.load_bytes(file.read())

Most entry types also support loading from other natural data types. Any data can be passed to the constructor directly and be delegated to the correct loader:

my_program = TIProgram("Disp \"HELLO WORLD!\"")
my_program.load_string("Disp \"HELLO WORLD!\"")

my_real = TIReal(1.23)
my_real.load_float(1.23)

Base TIEntry objects, as well other parent types like TIGDB, will be automatically coerced to the correct type:

# Coerces to a TIProgram
my_entry = TIEntry.open("HELLO.8xp")

Tip

Any entry type can be cast to any other by setting the object's __class__.

Exporting objects

Vars

Export a var as bytes or straight to a file:

my_var.save("HELLO.8xp")

# Infer the filename and extension
my_var.save()

with open("HELLO.8xp", 'wb+') as file:
    file.write(my_var.bytes())

Important

.save() uses the var's name as the filename, saving to the current working directory.

Entries

Entries can be passed an explicit header to attach or model to target when exporting:

my_program.save("HELLO.8xp")
my_program.save()

with open("HELLO.8xp", 'wb+') as file:
    file.write(my_program.export(header=my_header).bytes())

Any input data type can also be exported to:

assert my_program.string() == "Disp \"HELLO WORLD!\""

assert my_real.float() == 1.23

Tip

Built-in types can be exported to using the standard constructors, e.g. str(my_program).

Data Manipulation

Data sections

Vars are comprised of individual sections which represent different forms of data, split across the header and entries. The var itself also contains the total entry length and checksum sections, but these are read-only to prevent file corruption.

You can read and write to individual sections of an entry or header as their "canonical" type:

my_header.comment = "This is my (even cooler) comment!"
my_program.archived = True

assert my_program.type_id == 0x05

Data sections can also be other entry types:

my_gdb = TIGDB()
my_gdb.Xmin = TIReal(0)

assert my_gdb.Xmax == TIReal(10)

Each section is annotated with the expected type.

Tip

Data sections can accept any subtype of their expected type.

Raw containers

All vars store their data sections as raw bytes in the format interpreted by the calculator. Access any data section as a member of the .raw attribute to view and edit these bytes directly.

my_header.raw.comment = "This is my (even rawer) comment!".encode('utf-8')
my_program.raw.archived = b'\x80'

assert my_program.raw.type_id == b'\x05'

Warning

Edits to read-only bytes like the checksum are reset whenever any other data in the var is updated.

Models

All TI-82/83/84 series calcs are represented as TIModel objects stored in tivars.models. Each model contains its name, metadata, and features; use has on a TIFeature to check that a model has a given a feature. Models are also used to determine var file extensions and token sheets.

Flash Files

Flash files such as apps, OSes, and certificates can be loaded using the TIFlashHeader base class or its children. A flash file is composed of one to three headers (though usually only one); these are not to be confused with var headers. A flash header does not need to be "packaged" into a larger file format like an entry in a regular var; see TIFlashHeader.open and TIFlashHeader.save.

Tip

Loading flash files into a TIEntry probably won't work very well.

Other Functionalities

PIL

The tivars.PIL package can be used to interface with PIL, the Python Imaging Library. Simply import the package to register codecs for each of the TI image types. You can then open such images directly into a PIL Image:

from PIL import Image
from tivars.PIL import *

img = Image.open("Pic1.8ci")
img.show()

Tokenization

Functions to decode and encode strings into tokens can be found in tivars.tokenizer. These functions utilize the TI-Toolkit token sheets, which are kept as a submodule in tivars.tokens. Support currently exists for all models in the 82/83/84 series; PR's concerning the sheets themselves should be directed upstream.

Documentation

Library documentation can be found on GitHub Pages.

The var file format(s) and data sections can be found in a readable format on the repository wiki. Much of the information is copied from the TI-83 Link Guide, though has been updated to account for color models.

Note

The wiki is still a work-in-progress. Why not contribute a page?

Examples

You can find more sample code in examples that details common operations on each of the entry types. There are also examples for interfacing with popular external libraries (e.g. NumPy, PIL). Contributions welcome!

tivars_lib_py's People

Contributors

kg583 avatar logicaljoe avatar rpitasky avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

Forkers

rpitasky

tivars_lib_py's Issues

Name fields are weird

The interpretation of the name field varies heavily with the entry type. It is "most often" best to think of the name as a single token, and if not that then a font string, though there are exceptions:

  • For lists (001h and 00Dh), if the second byte is <6 it tokenizes it, if 040h it prints "IDList", else it just uses the font for up to 5 chars.
  • For programs, appvars, and groups (005h, 006h, 015h, and 017h), it displays a font cstr of up to 8 chars.
  • For Window settings (00Fh, 010h, and 011h), it displays a hardcoded string "Window", "RclWindow", or "TblSet" respectively (or a localized string)
  • And Images (01Ah), it tokenizes 0EF50h + [byte two].

(summarized by @LogicalJoe following lots of reverse engineering)

The lib currently treats every name as tokenized, which agrees with the font and some of the special cases for the most part but begins to fail for things like GDBs.

  • Lists tokenize for L1 - L6, have "IDList" for some reason, and then use the font otherwise (up to 5)
  • Programs, appvars, and groups use the font (up to 8)
  • Settings types use hardcoded strings
  • Images tokenize the first two bytes
  • Everything else tokenizes the first byte (new default)

Data-driven wiki build system

The current contents of wiki/build.py are a complete mess. There are two main contributors to this:

  • The documentation needs to extract elements of both the library modules and file ASTs, and currently combines them in a somewhat awkward way.
  • Many special cases are made to account for certain types, which should not live in-code.

Thus, a more data-driven formatter is in order. The module and AST extraction will simply dump all the components into a bucket, with the complexity of specifying how to arrange the components moved to page templates. This will in particular leave room for fetching class information that is invisible to the current system.

This build system will also be hooked into the publish-wiki GH action.

New token sheets

The library has been using the old format of the token sheets for a few months now (see the submodule commit currently used by tivars/tokenizer). This could be resolved immediately, but the sheets are getting updated again. Once this is complete, the library will be brought to use the latest version for the foreseeable future.

`pip` doesn't like submodules

The pip command included in the README doesn't play nice with the tokens submodule; trying to fix this isn't helped by the submodule targeting a very old commit (see #8). Once everything gets ushered along upstream, I'll rework the setuptools integration, and also probably push to PyPI for good measure.

Until then, clone this repo directly with Git to be sure you download everything.

Axe.xml not found

Installed with pip3 install git

[Errno 2] No such file or directory: 'C:\Users\********\AppData\Roaming\Python\Python311\site-packages\tivars\tokenizer\tokens/Axe.xml'

Format Specifiers

On top of having as many reasonable canonical string conversions as possible, plenty of types could leverage format specifiers that can be passed into str.format and f-strings.

  • f and d for numeric types, inheriting the arguments for regular floats and Decimals (e.g :.6f for six decimals of precision).
  • t for a variety of types, returning a string representation that can be pasted directly into a token editor (i.e. {1, 2, ~3} instead of [1, 2, -3]).

These formats should also be acceptable to load_string.

Edit: b has been removed due to lots of potential weirdness with encodings. It was pretty useless anyway.

GDBs are missing sections and JSON converters

A number of undocumented/underspecified sections of GDBs have since been specified. These new sections should be implemented accordingly.

Additionally, GDBs (and probably some other similar types) deserve a simple JSON format for loading and exporting, which can then be hooked into string/load_string and a new dict/load_dict. This format should match that used by tivars_lib_cpp for compatibility, with details TBD.

  • Document and implement the remaining GDB sections
  • Determine the canonical GDB JSON format
  • Implement string/load_string and dict/load_dict

Some wiki pages aren't quite up to snuff

Because no two types need quite the same information to be documented fairly, the current wiki build system is a tad lacking for some types. Some of this can be inserted into the build manually; the rest can be added into the lib as extra documentation.

  • Lists, matrices, tokenized types, and pictures need a generic data subsection describing what they contain.
  • GDBs need... attention. I'm not sure how much more.
  • Settings and GDBs should have subsections for their constant bytes.

Include all subsection format classes w/ cross-links

Currently, the wiki only documents children of TIEntry. This needs to be expanded to include any and all classes that describe subsection formats, such as enums (e.g. gdb/GraphColor), flags (e.g. numeric/FloatFlags), and certain non-obvious converters (e.g. numeric/BCD).

All subsection formats, meanwhile, need cross-links to their definitions.

Task List: Implemented Types

tivars_lib_py Type Implementation Progress

Checked items are fully implemented, though may yet have quality-of-life additions.
Types with partial implementations will be denoted with their current state of progress.

Types will be marked with their first release version when appropriate; if the release is in italics, then the release number is pending.

These types are canonical ports of those supported by tivars_lib_cpp, though may as of yet have fewer (or more) implemented interfaces. If you're unsure if the implementation here can meet your needs, head over to _cpp instead, though an issue identifying the disparity would be appreciated.

Basic Types - Types accessible by BASIC programs.

  • Real (v0.5)
  • RealList (v0.5)
  • Matrix (v0.5)
  • Equation (v0.5)
  • String (v0.5)
  • Program (v0.5)
  • ProtectedProgram (v0.5)
  • Picture (v0.6)
  • GDB (v0.5)
  • Complex (v0.5)
  • ComplexList (v0.5)
  • WindowSetttings (v0.5)
  • RecallWindow (v0.5)
  • TableSettings (v0.5)
  • RealFraction (v0.7)
  • Image (v0.6)

Exact Types - Types from the 83PCE and 82AEP

  • ComplexFraction (v0.7)
  • RealRadical (v0.7)
  • ComplexRadical (v0.7)
  • ComplexPi (v0.7)
  • ComplexPiFraction (v0.7)
  • RealPi (v0.7)
  • RealPiFraction (v0.7)

Non-Basic Types - Types that BASIC (usually) doesn't look at

  • Group (v0.8)
  • AppVar (v0.7)
  • OS (v0.9)
  • App (v0.9)
  • Certificates (v0.9)
  • License (v0.9)
  • Other Flash Types - they exist but are so unlikely to ever be floating around in a file

Any remaining types are not within the purview of the library until after the 1.0 release.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.