Add parakeet to examples/models #16349
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/16349
Note: Links to docs will display an error until the docs builds have been completed. ❌ 3 New Failures, 1 Unrelated FailureAs of commit 09cb1be with merge base 0ee2f49 ( NEW FAILURES - The following jobs have failed:
UNSTABLE - The following job is marked as unstable, possibly due to flakiness on trunk:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
bot
added
the
CLA Signed
This PR needs a
|
JacobSzwejbka
requested review from
kirklandsign,
larryliu0820 and
lucylq
as code owners
| # Find executorch libraries | ||
| list(APPEND CMAKE_FIND_ROOT_PATH ${CMAKE_CURRENT_BINARY_DIR}/../../..) | ||
| find_package(executorch CONFIG REQUIRED FIND_ROOT_PATH_BOTH) | ||
| executorch_target_link_options_shared_lib(executorch) |
There was a problem hiding this comment.
I don't think you need this
| if(CMAKE_TOOLCHAIN_FILE MATCHES ".*(iOS|ios\.toolchain)\.cmake$") | ||
| set(CMAKE_TOOLCHAIN_IOS ON) | ||
| else() | ||
| set(CMAKE_TOOLCHAIN_IOS OFF) | ||
| endif() |
|
|
||
| # Common ops for all builds | ||
| list(APPEND link_libraries optimized_native_cpu_ops_lib cpublas eigen_blas) | ||
| executorch_target_link_options_shared_lib(optimized_native_cpu_ops_lib) |
|
|
||
|
|
||
| def load_model(): | ||
| import nemo.collections.asr as nemo_asr |
There was a problem hiding this comment.
For this I think you need to add a install_requirements.txt in the folder so that people can install it
| std::vector<float> transposed_data(batch * time_steps * enc_dim); | ||
| const float* src = encoder_output.const_data_ptr<float>(); | ||
| for (int64_t t = 0; t < time_steps; t++) { | ||
| for (int64_t d = 0; d < enc_dim; d++) { | ||
| transposed_data[t * enc_dim + d] = src[d * time_steps + t]; | ||
| } | ||
| } |
There was a problem hiding this comment.
Interesting. Wondering why this can't be part of the model?
Specifically the parakeet-tdt-0.6b-v3 checkpoint
For the most part this was pretty fast and easy. 90% of the time was just me catching up on how the parakeet model was structured and what the decode loop looks like. An even better solution would be to merge this runner with the one we use for whisper asr.
Manually verified against no backend targeting (so Its kinda slow).