sun1638650145 / libraries-and-extensions-for-tensorflow-for-apple-silicon Goto Github PK

I have tensorflow-macos and tensorflow-metal v2.9.2 installed on my Macbook M1 with Python 3.10.5.

Are they the right version of Python and tensorflow installed for tensorflow-text?

The build of Text-2.90 failed with the following erros.

ERROR: /private/var/tmp/_bazel_fredwu/111899d0b2568b38b38ed1b7a930bdf7/external/com_google_sentencepiece/BUILD.bazel:51:11: Compiling src/normalizer.cc failed: (Exit 1): cc_wrapper.sh failed: error executing command external/local_config_cc/cc_wrapper.sh -U_FORTIFY_SOURCE -fstack-protector -Wall -Wthread-safety -Wself-assign -Wunused-but-set-parameter -Wno-free-nonheap-object -fcolor-diagnostics ... (remaining 70 arguments skipped)
external/com_google_sentencepiece/src/normalizer.cc:25:10: error: module @com_google_sentencepiece//:sentencepiece_processor does not depend on a module exporting 'third_party/darts_clone/darts.h'
#include "third_party/darts_clone/darts.h"

I also tried to use bazel 5.1.1, but it gave the error saying it required 5.3

ERROR: The project you're trying to build requires Bazel 5.3.0 (specified in /Users/xxx/Downloads/text-2.9.0/.bazelversion), but it wasn't found in /opt/homebrew/Cellar/bazel/5.1.1/libexec/bin.

And also the wheel could not be installed either.

ERROR: tensorflow_text-2.9.0-cp38-cp38-macosx_10_16_arm64.whl is not a supported wheel on this platform.

Any advises would be appreciated.

Hi,
thank you so much for these pre-compiled releases!
Do you plan to add tensorflow_text-*-cp*-cp*-macosx_13_0_arm64.whl builds?

Hello @sun1638650145 !
thank you for you great guide how to build lib on M1 Apple machines!

Following your instructions I was able to build the lib but when I try to load it in my notebook it fails with error

dlopen(/Users/xxxxxxx/conda/tf/lib/python3.9/site-packages/tensorflow_text/python/ops/_regex_split_ops.dylib, 0x0006): re-export ordinal -2 in /Users/xxxxxxx/conda/tf/lib/python3.9/site-packages/tensorflow_text/python/ops/_regex_split_ops.dylib out of range for __ZN10tensorflow14kernel_factory17OpKernelRegistrar12InitInternalEPKNS_9KernelDefENSt3__117basic_string_viewIcNS5_11char_traitsIcEEEENS5_10unique_ptrINS0_15OpKernelFactoryENS5_14default_deleteISB_EEEEsymbol not found in flat namespace '__ZN10tensorflow14kernel_factory17OpKernelRegistrar12InitInternalEPKNS_9KernelDefENSt3__117basic_string_viewIcNS5_11char_traitsIcEEEENS5_10unique_ptrINS0_15OpKernelFactoryENS5_14default_deleteISB_EEEE'

python --version
Python 3.9.13

conda list | grep tensorflow
tensorflow-addons 0.17.1 pypi_0 pypi
tensorflow-deps 2.9.0 0 apple
tensorflow-estimator 2.9.0 pypi_0 pypi
tensorflow-hub 0.12.0 pypi_0 pypi
tensorflow-macos 2.9.0 pypi_0 pypi
tensorflow-metal 0.5.0 pypi_0 pypi
tensorflow-text 2.9.0 pypi_0 pypi

nm /Users/xxxxxxx/conda/tf/lib/python3.9/site-packages/tensorflow_text/python/ops/_regex_split_ops.dylib | grep "__ZN10tensorflow14kernel_factory17OpKernelRegistrar12InitInternalEPKNS_9KernelDefENSt3__117basic_string_viewIcNS5_11char_traitsIcEEEENS5_10unique_ptrINS0_15OpKernelFactoryENS5_14default_deleteISB_EEEE"
U __ZN10tensorflow14kernel_factory17OpKernelRegistrar12InitInternalEPKNS_9KernelDefENSt3__117basic_string_viewIcNS5_11char_traitsIcEEEENS5_10unique_ptrINS0_15OpKernelFactoryENS5_14default_deleteISB_EEEE
I __ZN10tensorflow14kernel_factory17OpKernelRegistrar12InitInternalEPKNS_9KernelDefENSt3__117basic_string_viewIcNS5_11char_traitsIcEEEENS5_10unique_ptrINS0_15OpKernelFactoryENS5_14default_deleteISB_EEEE (indirect for __ZN10tensorflow14kernel_factory17OpKernelRegistrar12InitInternalEPKNS_9KernelDefENSt3__117basic_string_viewIcNS5_11char_traitsIcEEEENS5_10unique_ptrINS0_15OpKernelFactoryENS5_14default_deleteISB_EEEE)

Any help would be much appreciated

ERROR: tensorflow_text-2.15.0-cp310-cp310-macosx_11_0_arm64.whl is not a supported wheel on this platform.

Macbook M1 pro arm64

PyCharm 2023.3.4 (Professional Edition)
Build #PY-233.14475.56, built on February 26, 2024
Licensed to Universitas Negeri Malang
Subscription is active until December 7, 2024.
Runtime version: 17.0.10+1-b1087.17 aarch64
VM: OpenJDK 64-Bit Server VM by JetBrains s.r.o.
macOS 14.3.1
GC: G1 Young Generation, G1 Old Generation
Memory: 2048M
Cores: 8
Metal Rendering is ON
Registry:
  ide.experimental.ui=true

Hello,

I want to use the newest tensorflow-metal version but with it the dependency- hell starts freaking me out :-D.
I need tensorflow 2.11 for it but tensorflow-text is not compatible with this tensorflow version. As pip states: the latest version that is compatible is tensorflow-text 2.9.

Thx in advance <3.

I installed tensorflow 15 from the releases page like this:

pip install https://github.com/sun1638650145/Libraries-and-Extensions-for-TensorFlow-for-Apple-Silicon/releases/download/v2.15/tensorflow-2.15.0-cp311-cp311-macosx_14_0_arm64.whl

Tensorflow does not list any cpus:

>>> import tensorflow as tf
>>> tf.config.list_physical_devices("GPU")
[]

Running on a clean venv with python 3.11.1

Hi!

With the guidance of the instructions you've provided (big thanks!) I was able to build a wheel file but something still seems to be not quite right as I'm not able to install it.

When I run pip install ./*.whl I get the following error

Processing ./tensorflow_text-2.8.1-cp38-cp38-macosx_11_0_arm64.whl
Requirement already satisfied: tensorflow-hub>=0.8.0 in ./env/lib/python3.8/site-packages (from tensorflow-text==2.8.1) (0.12.0)
ERROR: Could not find a version that satisfies the requirement tensorflow<2.9,>=2.8.0 (from tensorflow-text) (from versions: none)
ERROR: No matching distribution found for tensorflow<2.9,>=2.8.0

python version: 3.8.10
tensorflow-macos: 2.8.0
tensorflow-metal: 0.5.0

tensorflow-text built from tag v2.8.1

Any thoughts on what might be the problem?

My trial had to stop at the compile step due to network issues. Can someone share a compiled file for me to install directly? My device is M1pro; TensorFlow 2.9.0

由于网络问题，我的试用不得不在编译的步骤上停止。有人能够分享给我一个编译好的文件以让我直接安装吗？我的设备是 M1pro；TensorFlow 2.9.0

Thanks for any help!
感谢您！

Is it planned to release a version of tf-text for the 2.13 release?

Hi, thanks for your builds and isnturctions, they are very useful.

The linux_aarch64 version is required to build tensorflow in docker on macOS .
I built tenserflow-text for linux_aarch64, maybe attach them to the release page? This will be helpful for anyone who has encountered this

tensorflow_text-2.15.0-cp39-11-cp39-11-linux_aarch64.whl.zip

Hi,

I am trying to use tf-text on Mac M2. Not clear if your repo provides the wheel or only tutorial. pip3 install tf-text / tf_text results in no matching distribution.

TIA

I am using RASA for my chatbot. It was working with Intel chipset. But for M1 it didnt work. I have searched internet alot. I follow them install tensorflow and numpy but my environment couldnt train my data. Then i searched internet and found this page. I followed the instructions and i hardly create my environment. Then when i press train button again error. Error looks like a mismatch between those packages:
`2022-10-04 12:39:55 ERROR softtechnlp.server - Traceback (most recent call last):
File "/Users/sadikalperbilgil/miniforge3/envs/envNLPV2arm2/lib/python3.9/site-packages/softtechnlp/server.py", line 1062, in train
training_result = await train_async(**training_payload)
File "/Users/sadikalperbilgil/miniforge3/envs/envNLPV2arm2/lib/python3.9/site-packages/softtechnlp/train.py", line 169, in train_async
return await train_async_internal(
File "/Users/sadikalperbilgil/miniforge3/envs/envNLPV2arm2/lib/python3.9/site-packages/softtechnlp/train.py", line 361, in train_async_internal
await do_training(
File "/Users/sadikalperbilgil/miniforge3/envs/envNLPV2arm2/lib/python3.9/site-packages/softtechnlp/train.py", line 407, in do_training
model_path = await train_nlu_with_validated_data(
File "/Users/sadikalperbilgil/miniforge3/envs/envNLPV2arm2/lib/python3.9/site-packages/softtechnlp/train.py", line 841, in train_nlu_with_validated_data
await softtechnlp.nlu.train(
File "/Users/sadikalperbilgil/miniforge3/envs/envNLPV2arm2/lib/python3.9/site-packages/softtechnlp/nlu/train.py", line 116, in train
interpreter = trainer.train(training_data, **kwargs)
File "/Users/sadikalperbilgil/miniforge3/envs/envNLPV2arm2/lib/python3.9/site-packages/softtechnlp/nlu/model.py", line 210, in train
updates = component.train(working_data, self.config, **context)
File "/Users/sadikalperbilgil/miniforge3/envs/envNLPV2arm2/lib/python3.9/site-packages/softtechnlp/sf/nlu/model.py", line 342, in train
super().train(training_data, config, **kwargs)
File "/Users/sadikalperbilgil/miniforge3/envs/envNLPV2arm2/lib/python3.9/site-packages/softtechnlp/nlu/classifiers/diet_classifier.py", line 832, in train
self.model.fit(
File "/Users/sadikalperbilgil/miniforge3/envs/envNLPV2arm2/lib/python3.9/site-packages/softtechnlp/utils/tensorflow/models.py", line 224, in fit
) = self.get_tf_train_functions(eager, model_data, batch_strategy)
File "/Users/sadikalperbilgil/miniforge3/envs/envNLPV2arm2/lib/python3.9/site-packages/softtechnlp/utils/tensorflow/models.py", line 489, in get_tf_train_functions
self.get_tf_call_model_function(
File "/Users/sadikalperbilgil/miniforge3/envs/envNLPV2arm2/lib/python3.9/site-packages/softtechnlp/utils/tensorflow/models.py", line 472, in get_tf_call_model_function
tf_call_model_function(next(iter(init_dataset)))
File "/Users/sadikalperbilgil/miniforge3/envs/envNLPV2arm2/lib/python3.9/site-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/var/folders/3r/lscxbw557b37671_q_xn6d300000gp/T/autograph_generated_file9g8h7p4r.py", line 11, in tf__train_on_batch
prediction_loss = ag.converted_call(ag.ld(self).batch_loss, (ag.ld(batch_in),), None, fscope)
File "/var/folders/3r/lscxbw557b37671_q_xn6d300000gp/T/autograph_generated_fileubdqh00g.py", line 23, in tf__batch_loss
(text_transformed, text_in, text_seq_ids, lm_mask_bool_text, ) = ag.converted_call(ag.ld(self).create_sequence, (ag.ld(tf_batch_data)[ag.ld(TEXT)][ag.ld(SEQUENCE)], ag__.ld(tf_batch_data)[ag__.ld(TEXT)][ag__.ld(SENTENCE)], ag__.ld(mask_sequence_text), ag__.ld(mask_text), ag__.ld(self).text_name), dict(sparse_dropout=ag__.ld(self).config[ag__.ld(SPARSE_INPUT_DROPOUT)], dense_dropout=ag__.ld(self).config[ag__.ld(DENSE_INPUT_DROPOUT)], masked_lm_loss=ag__.ld(self).config[ag__.ld(MASKED_LM)], sequence_ids=True), fscope)
File "/var/folders/3r/lscxbw557b37671_q_xn6d300000gp/T/autograph_generated_filemw2bj9mv.py", line 27, in tf___create_sequence
inputs = ag.converted_call(ag__.ld(self).combine_sequence_sentence_features, (ag_.ld(sequence_features), ag__.ld(sentence_features), ag__.ld(mask_sequence), ag__.ld(mask), ag__.ld(name), ag__.ld(sparse_dropout), ag__.ld(dense_dropout)), None, fscope)
File "/var/folders/3r/lscxbw557b37671_q_xn6d300000gp/T/autograph_generated_file2l1w562x.py", line 10, in tf___combine_sequence_sentence_features
sequence_x = ag.converted_call(ag__.ld(self).combine_sparse_dense_features, (ag_.ld(sequence_features), f'{ag__.ld(name)}{ag__.ld(SEQUENCE)}', ag__.ld(mask_sequence), ag__.ld(sparse_dropout), ag__.ld(dense_dropout)), None, fscope)
File "/var/folders/3r/lscxbw557b37671_q_xn6d300000gp/T/autograph_generated_filedqt8w5sf.py", line 119, in tf___combine_sparse_dense_features
ag.if_stmt(ag__.not(ag__.ld(features)), if_body_4, else_body_4, get_state_5, set_state_5, ('do_return', 'retval_'), 2)
File "/var/folders/3r/lscxbw557b37671_q_xn6d300000gp/T/autograph_generated_filedqt8w5sf.py", line 88, in else_body_4
ag.for_stmt(ag__.ld(features), None, loop_body, get_state_3, set_state_3, (), {'iterate_names': 'f'})
File "/var/folders/3r/lscxbw557b37671_q_xn6d300000gp/T/autograph_generated_filedqt8w5sf.py", line 84, in loop_body
ag.if_stmt(ag__.converted_call(ag__.ld(isinstance), (ag__.ld(f), ag__.ld(tf).SparseTensor), None, fscope), if_body_2, else_body_2, get_state_2, set_state_2, (), 0)
File "/var/folders/3r/lscxbw557b37671_q_xn6d300000gp/T/autograph_generated_filedqt8w5sf.py", line 62, in if_body_2
ag.if_stmt(ag__.ld(sparse_dropout), if_body, else_body, get_state, set_state, ('f',), 1)
File "/var/folders/3r/lscxbw557b37671_q_xn6d300000gp/T/autograph_generated_filedqt8w5sf.py", line 57, in if_body
f = ag.converted_call(ag.ld(self).tf_layers[f'sparse_input_dropout.{ag_.ld(name)}'], (ag__.ld(f), ag__.ld(self).training), None, fscope)
File "/Users/sadikalperbilgil/miniforge3/envs/envNLPV2arm2/lib/python3.9/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/var/folders/3r/lscxbw557b37671_q_xn6d300000gp/T/autograph_generated_fileqluryb_j.py", line 67, in tf__call
outputs = ag.converted_call(ag_.ld(tf_utils).smart_cond, (ag__.ld(training), ag__.ld(dropped_inputs), ag__.autograph_artifact(lambda : ag__.converted_call(ag__.ld(tf).identity, (ag__.ld(inputs),), None, fscope))), None, fscope)
AttributeError: in user code:

File "/Users/sadikalperbilgil/miniforge3/envs/envNLPV2arm2/lib/python3.9/site-packages/softtechnlp/utils/tensorflow/models.py", line 298, in train_on_batch  *
    prediction_loss = self.batch_loss(batch_in)
File "/Users/sadikalperbilgil/miniforge3/envs/envNLPV2arm2/lib/python3.9/site-packages/softtechnlp/nlu/classifiers/diet_classifier.py", line 1448, in batch_loss  *
    (
File "/Users/sadikalperbilgil/miniforge3/envs/envNLPV2arm2/lib/python3.9/site-packages/softtechnlp/utils/tensorflow/models.py", line 1066, in _create_sequence  *
    inputs = self._combine_sequence_sentence_features(
File "/Users/sadikalperbilgil/miniforge3/envs/envNLPV2arm2/lib/python3.9/site-packages/softtechnlp/utils/tensorflow/models.py", line 960, in _combine_sequence_sentence_features  *
    sequence_x = self._combine_sparse_dense_features(
File "/Users/sadikalperbilgil/miniforge3/envs/envNLPV2arm2/lib/python3.9/site-packages/softtechnlp/utils/tensorflow/models.py", line 928, in _combine_sparse_dense_features  *
    _f = self._tf_layers[f"sparse_input_dropout.{name}"](
File "/Users/sadikalperbilgil/miniforge3/envs/envNLPV2arm2/lib/python3.9/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler  **
    raise e.with_traceback(filtered_tb) from None
File "/var/folders/3r/lscxbw557b37671_q_xn6d300000gp/T/__autograph_generated_fileqluryb_j.py", line 67, in tf__call
    outputs = ag__.converted_call(ag__.ld(tf_utils).smart_cond, (ag__.ld(training), ag__.ld(dropped_inputs), ag__.autograph_artifact(lambda : ag__.converted_call(ag__.ld(tf).identity, (ag__.ld(inputs),), None, fscope))), None, fscope)

AttributeError: Exception encountered when calling layer "sparse_dropout_1" (type SparseDropout).

in user code:

    File "/Users/sadikalperbilgil/miniforge3/envs/envNLPV2arm2/lib/python3.9/site-packages/softtechnlp/utils/tensorflow/layers.py", line 64, in call  *
        outputs = tf_utils.smart_cond(

    AttributeError: module 'tensorflow.python.keras.utils.tf_utils' has no attribute 'smart_cond'


Call arguments received by layer "sparse_dropout_1" (type SparseDropout):
  • inputs=<tensorflow.python.framework.sparse_tensor.SparseTensor object at 0x2c2d6d640>
  • training=True

2022-10-04 12:39:55 ERROR softtechnlp.server - An unexpected error occurred during training. Error: in user code:

File "/Users/sadikalperbilgil/miniforge3/envs/envNLPV2arm2/lib/python3.9/site-packages/softtechnlp/utils/tensorflow/models.py", line 298, in train_on_batch  *
    prediction_loss = self.batch_loss(batch_in)
File "/Users/sadikalperbilgil/miniforge3/envs/envNLPV2arm2/lib/python3.9/site-packages/softtechnlp/nlu/classifiers/diet_classifier.py", line 1448, in batch_loss  *
    (
File "/Users/sadikalperbilgil/miniforge3/envs/envNLPV2arm2/lib/python3.9/site-packages/softtechnlp/utils/tensorflow/models.py", line 1066, in _create_sequence  *
    inputs = self._combine_sequence_sentence_features(
File "/Users/sadikalperbilgil/miniforge3/envs/envNLPV2arm2/lib/python3.9/site-packages/softtechnlp/utils/tensorflow/models.py", line 960, in _combine_sequence_sentence_features  *
    sequence_x = self._combine_sparse_dense_features(
File "/Users/sadikalperbilgil/miniforge3/envs/envNLPV2arm2/lib/python3.9/site-packages/softtechnlp/utils/tensorflow/models.py", line 928, in _combine_sparse_dense_features  *
    _f = self._tf_layers[f"sparse_input_dropout.{name}"](
File "/Users/sadikalperbilgil/miniforge3/envs/envNLPV2arm2/lib/python3.9/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler  **
    raise e.with_traceback(filtered_tb) from None
File "/var/folders/3r/lscxbw557b37671_q_xn6d300000gp/T/__autograph_generated_fileqluryb_j.py", line 67, in tf__call
    outputs = ag__.converted_call(ag__.ld(tf_utils).smart_cond, (ag__.ld(training), ag__.ld(dropped_inputs), ag__.autograph_artifact(lambda : ag__.converted_call(ag__.ld(tf).identity, (ag__.ld(inputs),), None, fscope))), None, fscope)

AttributeError: Exception encountered when calling layer "sparse_dropout_1" (type SparseDropout).

in user code:

    File "/Users/sadikalperbilgil/miniforge3/envs/envNLPV2arm2/lib/python3.9/site-packages/softtechnlp/utils/tensorflow/layers.py", line 64, in call  *
        outputs = tf_utils.smart_cond(

    AttributeError: module 'tensorflow.python.keras.utils.tf_utils' has no attribute 'smart_cond'


Call arguments received by layer "sparse_dropout_1" (type SparseDropout):
  • inputs=<tensorflow.python.framework.sparse_tensor.SparseTensor object at 0x2c2d6d640>
  • training=True

I have my toml file for the project like: build-system]
requires = [ "poetry-core>=1.0.0",]
build-backend = "poetry.core.masonry.api"

[tool.black]
line-length = 88
target-version = [ "py36", "py37", "py38","py39"]
exclude = "((.eggs | .git | .pytest_cache | build | dist))"

[tool.poetry]
name = "softtechnlp"
version = "2.3.3.2.dev"
description = "Open source machine learning framework to automate text- and voice-based conversations: NLU, dialogue management, connect to Slack, Facebook, and more - Create chatbots and voice assistants"
authors = [ "",]
maintainers = [ "",]
homepage = ""
repository = ""
documentation = ""
classifiers = [ "Development Status :: 4 - Beta", "Intended Audience :: Developers", "License :: OSI Approved :: Apache Software License", "Programming Language :: Python", "Programming Language :: Python :: 3.6", "Programming Language :: Python :: 3.7", "Programming Language :: Python :: 3.8", "Topic :: Software Development :: Libraries",]
keywords = [ "nlp", "machine-learning", "machine-learning-library", "bot", "bots", "botkit", "conversational-agents", "conversational-ai", "chatbot", "chatbot-framework", "bot-framework",]
include = [ "LICENSE.txt", "README.md",]
readme = "README.md"
license = "Apache-2.0"

[tool.towncrier]
package = "softtechnlp"
package_dir = "softtechnlp"
filename = "CHANGELOG.mdx"
directory = "./changelog"
underlines = " "
title_format = "## [{version}] - {project_date}"
template = "./changelog/_template.md.jinja2"
start_string = "\n"
issue_format = ""
[[tool.towncrier.type]]
directory = "removal"
name = "Deprecations and Removals"
showcontent = true

[[tool.towncrier.type]]
directory = "feature"
name = "Features"
showcontent = true

[[tool.towncrier.type]]
directory = "improvement"
name = "Improvements"
showcontent = true

[[tool.towncrier.type]]
directory = "bugfix"
name = "Bugfixes"
showcontent = true

[[tool.towncrier.type]]
directory = "doc"
name = "Improved Documentation"
showcontent = true

[[tool.towncrier.type]]
directory = "misc"
name = "Miscellaneous internal changes"
showcontent = false

[tool.poetry.dependencies]
python = ">=3.6,<3.10"
boto3 = "^1.12"
requests = "^2.23"
requests_futures = "^1.0.0"
fuzzy_matcher = "^0.1.0"
fuzzywuzzy = "0.18.0"
sgqlc = "^14.1"
pypred = { git = "https://[email protected]/dialoguemd/pypred.git", rev = "7e30c9078e8a34a4ba3ecf96c6ea826173b25063" }
matplotlib = ">=3.1,<3.4"
attrs = ">=19.3,<20.4"
jsonpickle = ">=1.3,<1.6"
redis = "^3.4"
numpy = [{version = ">=1.23", markers = "sys_platform!='darwin'"},{version = "=1.19.5", markers = "sys_platform=='darwin'"}]
scipy = "^1.4.1"
absl-py = ">=0.9,<0.12"
apscheduler = ">=3.6,<3.8"
tqdm = ">=4.31,<4.57"
networkx = ">=2.4,<2.6"
fbmessenger = "~6.0.0"
pykwalify = ">=1.7,<1.9"
coloredlogs = ">=10,<15"
"ruamel.yaml" = "^0.16.5"
scikit-learn = { version = ">=0.22,<0.25", markers="platform_machine != 'arm64'"}
slackclient = "^2.0.0"
twilio = ">=6.26,<6.51"
webexteamssdk = ">=1.1.1,<1.7.0"
mattermostwrapper = "~2.2"
rocketchat_API = ">=0.6.31,<1.10.0"
colorhash = "~1.0.2"
jsonschema = "~3.2"
packaging = ">=20.0,<21.0"
pytz = ">=2019.1,<2021.0"
softtechnlp-sdk = "^2.3.1"
colorclass = "~2.2"
terminaltables = "~3.1.0"
sanic = ">=19.12.2,<21.0.0"
sanic-cors = "~0.10.0b1"
sanic-jwt = ">=1.3.2,<2.0"
cloudpickle = ">=1.2,<1.7"
multidict = "^4.6"
aiohttp = "~3.6"
questionary = "~1.5.1"
prompt-toolkit = "^2.0"
python-socketio = ">=5,<6"
python-engineio = ">=4,<5"
pydot = "~1.4"
async_generator = "~1.10"
SQLAlchemy = "~1.3.3"
sklearn-crfsuite = "~0.3"
psycopg2-binary = "~2.8.2"
python-dateutil = "~2.8"
tensorflow = { version = "~2.8.2", markers="platform_machine != 'arm64'"}
tensorflow-text = [{ version = "~2.8.0", markers = "sys_platform!='win32' and sys_platform!='darwin'"}]
tensorflow_hub = [{ version = "~2.8.0", markers = "sys_platform!='win32' and sys_platform!='darwin'"}]
tensorflow-addons = [{version = "~0.10", markers="sys_platform!='darwin'"},]
tensorflow-estimator = [{version = "~2.6", markers="sys_platform!='darwin'"},]
tensorflow-probability = [{version = "~0.11", markers="sys_platform!='darwin'"},]
setuptools = ">=41.0.0"
kafka-python = ">=1.4,<3.0"
ujson = ">=1.35,<5.0"
oauth2client = "4.1.3"
regex = ">=2020.6,<2020.10"
joblib = "^0.15.1"
sentry-sdk = ">=0.17.0,<0.20.0"
aio-pika = "^6.7.1"
pyTelegramBotAPI = "^3.7.3"
prometheus-client = "^0.8.0"
instana = "^1.37.4"
python-dotenv = "^0.20.0"
fasttext = "^0.9.2"
spacymoji = "2.0.0"
spacy = { version = "2.3.0", markers="sys_platform!='darwin'"}
grpcio= ">=1.45.0"

[tool.poetry.dev-dependencies]
pytest-cov = "^2.10.0"
pytest-localserver = "^0.5.0"
pytest-sanic = "^1.6.1"
pytest-asyncio = "^0.10.0"
pytest-xdist = "^1.32.0"
pytest = "^5.3.4"
freezegun = "^1.0.0"
responses = "^0.12.1"
aioresponses = "^0.6.2"
moto = "~=1.3.16"
fakeredis = "^1.4.0"
mongomock = "^3.18.0"
black = "^19.10b0"
flake8 = "^3.8.3"
flake8-docstrings = "^1.5.0"
google-cloud-storage = "^1.29.0"
azure-storage-blob = "<12.6.0"
coveralls = "^2.0.0"
towncrier = "^19.2.0"
toml = "^0.10.0"
pep440-version-utils = "^0.3.0"
pydoc-markdown = "^3.5.0"
pytest-timeout = "^1.4.2"
mypy = "^0.790"
bandit = "^1.6.3"

[tool.poetry.extras]
jieba = [ "jieba",]
transformers = [ "transformers",]
full = [ "transformers", "jieba",]
gh-release-notes = [ "github3.py",]

[tool.poetry.scripts]
softtechnlp = "softtechnlp.main:main"

[tool.poetry.dependencies.PyJWT]
version = "^2.0.0"
extras = [ "crypto",]

[tool.poetry.dependencies.colorama]
version = "^0.4.4"
markers = "sys_platform == 'win32'"

[tool.poetry.dependencies."github3.py"]
version = "~1.3.0"
optional = true

[tool.poetry.dependencies.transformers]
version = ">=2.4,<2.12"
optional = true

[tool.poetry.dependencies.jieba]
version = ">=0.39, <0.43"
optional = true

[tool.poetry.dependencies.pymongo]
version = ">=3.8,<3.11"
extras = [ "tls", "srv",]
`

As you can see i installed tensorflow-macos metal addons and text manually with the instructions on this page. Now i am stuck.

Can you have any insight?

downloaded latest version of package and tried to install it but it's not successful. it complains,
pip install tensorflow_text-2.12.1-cp39-cp39-macosx_11_0_arm64.whl
ERROR: tensorflow_text-2.12.1-cp39-cp39-macosx_11_0_arm64.whl is not a supported wheel on this platform.

Following is my current environment:
conda list | grep -i tensorflow
tensorflow-deps 2.9.0 0 apple
tensorflow-estimator 2.12.0 pypi_0 pypi
tensorflow-macos 2.12.0 pypi_0 pypi
tensorflow-metal 0.8.0 pypi_0 pypi

python --version
Python 3.10.11

I see "cp310" vs cp311 vs cp39 in the wheels.
Sorry to be ignorant, but how do these vary?

hi

so with your help on my previous issue I were able to build and install the tensorflow-text whl.

However, I noticed when continuing working on this, that I still seem to have some problems when importing and using tensorflow-text. This issue do not happen when I install and use the wheel that you have provided.

Any ideas on what may be the issue?

from tensorflow_text.core.pybinds import tflite_registrar
E   ImportError: dlopen(python3.8/site-packages/tensorflow_text/core/pybinds/tflite_registrar.so, 0x0002): tried: 'python3.8/site-packages/tensorflow_text/core/pybinds/tflite_registrar.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e'))

First off, I applaud you for maintaining these instructions for people struggling with getting things running on an M1 machine...

Following you instructions for building tensorflow_text, I run into the following issue:

ERROR: /private/var/tmp/_bazel_USER/9f8724fe94ef5d3d184dee8c41f63e97/external/com_google_protobuf/BUILD:981:21: in proto_lang_toolchain rule @com_google_protobuf//:cc_toolchain: '@com_google_protobuf//:cc_toolchain' does not have mandatory provider 'ProtoInfo'.

I tried various suggestions on modifying the WORKPLACE file, but haven't been able to get around it....

I created a new environment to install tensorflow and tensorflow-text step by step on Macbook M1 for Python==3.11.

However, building tensorflow-text encounted the errors

fatal error: too many errors emitted, stopping now [-ferror-limit=]
1 warning and 20 errors generated.
Target //oss_scripts/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 4.042s, Critical Path: 1.43s
INFO: 55 processes: 20 internal, 35 local.
FAILED: Build did NOT complete successfully

Any help would be welcomed.

Hello! Could you update tf-text to 2.15?

Hello,

following exactly the instructions (w/ python 3.9) I keep getting a compiler error that I don't know how to interpret/resolve.

ERROR: /private/var/tmp/_bazel_XXXXX/03c196188fbb5dc38e3a61a403354856/external/com_google_sentencepiece/BUILD.bazel:51:11: Compiling src/normalizer.cc failed: (Exit 1): cc_wrapper.sh failed: error executing command external/local_config_cc/cc_wrapper.sh -U_FORTIFY_SOURCE -fstack-protector -Wall -Wthread-safety -Wself-assign -Wunused-but-set-parameter -Wno-free-nonheap-object -fcolor-diagnostics ... (remaining 70 arguments skipped)
external/com_google_sentencepiece/src/normalizer.cc:25:10: error: module: @com_google_sentencepiece//:sentencepiece_processor does not depend on a module exporting 'third_party/darts_clone/darts.h'
#include "third_party/darts_clone/darts.h"
         ^
1 error generated.
Target //oss_scripts/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.

Platform: M1 Pro with Monterey (12.4)

If anyone has an idea where this might come from, any help is greatly appreciated.

Hello there!
Congrats for the progress on your reselases for tensorflow-text library for MAC M1 .
These add a significant value to the community! 🥇 💯

I was able to follow this tutorial thanks to your release , previous to this one, at apply transformers for Neural Machine translation with python 3.9 an tensorflow-mac-os 2.8 for creating tokenized datasets by installing one of your releases. Thanks so much!

However to advance with decoder mask multihead attention setup for Transformers tutorial and call one of its arguments use_causal_mask=True , Tensorflow 2.10 is required , and I think I need tensorflow-text for 2.10. I saw that you have the addons wheel but not the library .whl itself ,

any idea on when it might be available or any workaround to track this to make it work?

Please don´t hesitate to point me out in the right direction if there´s any worng assuption here...

For installing TensorFlow 2.10 , I´ve followed this :

conda install -c apple tensorflow-deps
conda install -c apple tensorflow-deps==2.10.0
python -m pip install tensorflow-macos

After that I can see that Tensorflow 2.10.0 has been installed.

Thanks so much for the time dedicated to this and keep up the great work! 👍

First off, thanks for writing this tutorial!

The process seemed to go okay, but I'm getting an error when trying to import tensorflow_text in python:

>>> import tensorflow_text as text
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/jbmaxwell/src/text-2.8.2/tensorflow_text/__init__.py", line 20, in <module>
    from tensorflow_text.core.pybinds import tflite_registrar
ImportError: cannot import name 'tflite_registrar' from 'tensorflow_text.core.pybinds' (unknown location)

It seems like a problem with the install.

I tried re-launching my shell session, but then I just get a ModuleNotFoundError: No module named 'tensorflow_text', as if it was never installed. Any idea what might be going on? Is there somewhere in particular I should run the bazel and text installs?

PS -- I'm running macOS 12.3, Xcode 13.2.1, on a 16" MacBook Pro M1 Pro.

Hi, thanks so much for creating this repo!

When I installed the provided tensorflow-text (pip install tensorflow-text@https://github.com/sun1638650145/Libraries-and-Extensions-for-TensorFlow-for-Apple-Silicon/releases/download/v2.14/tensorflow_text-2.14.0-cp39-cp39-macosx_11_0_arm64.whl), it is working as expected.

However, when I tried to compile it myself following instructions at https://github.com/sun1638650145/Libraries-and-Extensions-for-TensorFlow-for-Apple-Silicon/blob/main/tutorials/text/text.md, I get the following error when trying to use it:

../../miniforge3/envs/.../lib/python3.9/site-packages/tensorflow/python/framework/load_library.py:54: in load_op_library
    lib_handle = py_tf.TF_LoadLibrary(library_filename)
E   tensorflow.python.framework.errors_impl.NotFoundError: dlopen(/Users/.../miniforge3/envs/.../lib/python3.9/site-packages/tensorflow_text/python/ops/_regex_split_ops.dylib, 0x0006): malformed trie child, cycle to nodeOffset=0x2
E   weak-def symbol not found '__ZN10tensorflow4text12RegexSplitOp14CachedDelimRE2ERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEE'

I wonder if you've seen this error before?