Integrate UrbanSoundDataset for Audio Data Processing by codingwithsurya · Pull Request #1179 · DSGT-DLP/Deep-Learning-Playground

codingwithsurya · 2024-05-14T02:09:02Z

adding urbansound-dataset and schemas.py

Github Issue Number Here: <ntegrate UrbanSoundDataset for Audio Trainspace #1156>
What user problem are we solving?
We are enhancing the Deep Learning Playground's capabilities to include audio data processing by integrating the UrbanSound8K dataset. This allows users to work with audio data seamlessly within the existing pipeline, expanding the versatility and application of the platform.

What solution does this PR provide?
This PR adds a new class, UrbanSoundDataset, to the training/core/dataset.py module. This class encapsulates functionalities for data ingestion, preprocessing, and loading specifically tailored for the UrbanSound8K dataset. It includes dataCreator, train_loader, and test_loader methods to facilitate efficient data loading into the model for training and testing. Additionally, it ensures compatibility with PyTorch's DataLoader mechanism and integrates smoothly with the existing training pipeline.

It also provides a schemas.py file that provides audio params. This file is still a WIP.

Testing Methodology

How did you test your changes and verify that existing functionality is not broken
manual testing

Any other considerations
Updated schemas.py to include AudioParams for defining non-tunable parameters specific to the UrbanSound8K dataset.

we also added 2 new dependencies to poetry -- torchaudio and soundata

sonarqubecloud · 2024-05-14T02:10:09Z

Quality Gate passed

Issues
10 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

codingwithsurya · 2024-05-14T02:14:23Z

looks like one of the lints are failing. weird. @karkir0003 @DSGT-DLP/project-lead

the error message is this:
Note: This error originates from the build backend, and is likely not a problem with poetry but with simpleaudio (1.0.4) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "simpleaudio (==1.0.4)"'.
Error: Process completed with exit code 1.

We just added torchaudio and soundata libraries in this pr to poetry so bc of that.

karkir0003

good start for the most part.

please address my questions and add unit tests to ensure the data loading logic works as intended

karkir0003 · 2024-05-14T02:12:40Z

+
+        soundData = tempData
+        soundFormatted = torch.zeros([32000, 1])
+        soundFormatted[:32000] = soundData[::5]  # Take every fifth sample of soundData


the reason why we are taking every fifth sample from soundData and assigning it to the first 32000 elements of soundFormatted is to downsample the data if soundData is a high-frequency sound signal.

ok. might want to clarify that

karkir0003 · 2024-05-14T02:18:05Z

looks like one of the lints are failing. weird. @karkir0003 @DSGT-DLP/project-lead

the error message is this:
Note: This error originates from the build backend, and is likely not a problem with poetry but with simpleaudio (1.0.4) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "simpleaudio (==1.0.4)"'.
Error: Process completed with exit code 1.

We just added torchaudio and soundata libraries in this pr to poetry so bc of that.

whats simpleaudio? is this a lib thats a dependency used by soundata?

karkir0003 · 2024-05-14T02:18:28Z

did we install the latest stable version of torchaudio and soundata?

karkir0003 · 2024-05-14T02:25:51Z

looks like one of the lints are failing. weird. @karkir0003 @DSGT-DLP/project-lead

the error message is this: Note: This error originates from the build backend, and is likely not a problem with poetry but with simpleaudio (1.0.4) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "simpleaudio (==1.0.4)"'. Error: Process completed with exit code 1.

We just added torchaudio and soundata libraries in this pr to poetry so bc of that.

I'd try starting the debugging with the following:

Try running the command that's shown in the log. See if any further logs come up
Maybe we might need to find a compatible version of soundata or potentially add simpleaudio as a dependency just like how you used DLP CLI to install torchaudio
If 1 and 2 don't work, try asking in the poetry github repo by filing a github issue. There's also a discord server for Poetry that can help clarify

codingwithsurya · 2024-05-14T02:30:53Z

looks like one of the lints are failing. weird. @karkir0003 @DSGT-DLP/project-lead
the error message is this:
Note: This error originates from the build backend, and is likely not a problem with poetry but with simpleaudio (1.0.4) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "simpleaudio (==1.0.4)"'.
Error: Process completed with exit code 1.
We just added torchaudio and soundata libraries in this pr to poetry so bc of that.

whats simpleaudio? is this a lib thats a dependency used by soundata?

still tryna figure this one out. i went through the logs and couldnt find any hints

codingwithsurya · 2024-05-14T02:31:28Z

did we install the latest stable version of torchaudio and soundata?

yup i double checked. i just ran dlp-cli backend add ____

codingwithsurya · 2024-05-14T02:32:26Z

looks like one of the lints are failing. weird. @karkir0003 @DSGT-DLP/project-lead
the error message is this: Note: This error originates from the build backend, and is likely not a problem with poetry but with simpleaudio (1.0.4) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "simpleaudio (==1.0.4)"'. Error: Process completed with exit code 1.
We just added torchaudio and soundata libraries in this pr to poetry so bc of that.

I'd try starting the debugging with the following:

Try running the command that's shown in the log. See if any further logs come up

Maybe we might need to find a compatible version of soundata or potentially add simpleaudio as a dependency just like how you used DLP CLI to install torchaudio

If 1 and 2 don't work, try asking in the poetry github repo by filing a github issue. There's also a discord server for Poetry that can help clarify

alright bet sounds good

karkir0003 · 2024-05-17T01:39:41Z

@codingwithsurya , looks like someone from poetry responded to this thread I created and shared with you in discord: https://github.com/orgs/python-poetry/discussions/9418

codingwithsurya · 2024-05-18T00:06:38Z

https://github.com/orgs/python-poetry/discussions/9418

ok. we can add it as a dependency then.

karkir0003 · 2024-05-18T01:23:38Z

https://github.com/orgs/python-poetry/discussions/9418

ok. we can add it as a dependency then.

let's give this a try and see if that works @codingwithsurya

adding urbansound-dataset and schemas.py

29ab872

codingwithsurya requested a review from a team as a code owner May 14, 2024 02:09

codingwithsurya linked an issue May 14, 2024 that may be closed by this pull request

[FEATURE]: Integrate UrbanSoundDataset for Audio Trainspace #1156

Open

karkir0003 reviewed May 14, 2024

View reviewed changes

Uh oh!

Conversation

codingwithsurya commented May 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

adding urbansound-dataset and schemas.py

Uh oh!

sonarqubecloud Bot commented May 14, 2024

Quality Gate passed

Uh oh!

codingwithsurya commented May 14, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

karkir0003 left a comment

Choose a reason for hiding this comment

Uh oh!

karkir0003 May 14, 2024

Choose a reason for hiding this comment

Uh oh!

codingwithsurya May 14, 2024

Choose a reason for hiding this comment

Uh oh!

karkir0003 May 15, 2024

Choose a reason for hiding this comment

Uh oh!

codingwithsurya May 15, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

karkir0003 commented May 14, 2024

Uh oh!

karkir0003 commented May 14, 2024

Uh oh!

karkir0003 commented May 14, 2024

Uh oh!

codingwithsurya commented May 14, 2024

Uh oh!

codingwithsurya commented May 14, 2024

Uh oh!

codingwithsurya commented May 14, 2024

Uh oh!

karkir0003 commented May 17, 2024

Uh oh!

codingwithsurya commented May 18, 2024

Uh oh!

karkir0003 commented May 18, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

codingwithsurya commented May 14, 2024 •

edited

Loading

codingwithsurya commented May 14, 2024 •

edited

Loading