Giter Club home page Giter Club logo

Comments (2)

ZachNagengast avatar ZachNagengast commented on May 23, 2024

Prompting is a little tricky with whisper, much different from LLMs, because the prompt is just what the model is supposed to assume is the transcription from the previous window. Here is a more in depth guide: https://cookbook.openai.com/examples/whisper_prompting_guide

To put it simply, you're usually best off giving a really good example of text that you want it to output as the response, and the model will try to follow that format. See more here in these unit tests:

WhisperKit/Tests/WhisperKitTests/UnitTests.swift

Lines 681 to 710 in 5572cd6

func testPromptTokens() async throws {
let whisperKit = try await WhisperKit(modelFolder: tinyModelPath(), verbose: true, logLevel: .debug)
let promptText = " prompt to encourage output without any punctuation and without capitalizing americans as if it was already normalized"
let tokenizer = whisperKit.tokenizer!
let promptTokens = tokenizer.encode(text: promptText).filter { $0 < tokenizer.specialTokens.specialTokenBegin }
let options = DecodingOptions(skipSpecialTokens: true, promptTokens: promptTokens)
let result = try await XCTUnwrapAsync(
await transcribe(with: .tiny, options: options),
"Failed to transcribe"
)
XCTAssertEqual(result.segments.first?.text, " and so my fellow americans ask not what your country can do for you ask what you can do for your country.")
}
func testPrefixTokens() async throws {
let whisperKit = try await WhisperKit(modelFolder: tinyModelPath(), verbose: true, logLevel: .debug)
// Prefix to encourage output without any punctuation and without capitalizing americans as if it was already normalized
let prefixText = " and so my fellow americans"
let tokenizer = whisperKit.tokenizer!
let prefixTokens = tokenizer.encode(text: prefixText).filter { $0 < tokenizer.specialTokens.specialTokenBegin }
let options = DecodingOptions(skipSpecialTokens: true, prefixTokens: prefixTokens)
let result = try await XCTUnwrapAsync(
await transcribe(with: .tiny, options: options),
"Failed to transcribe"
)
XCTAssertEqual(result.segments.first?.text, " and so my fellow americans ask not what your country can do for you ask what you can do for your country.")
}

from whisperkit.

YiLee01 avatar YiLee01 commented on May 23, 2024

Thank you for your help, you solved my problem!

from whisperkit.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.