Skip to content
This repository was archived by the owner on Oct 16, 2020. It is now read-only.

python3 initial#9

Open
krflol wants to merge 10 commits into
CREEATION:masterfrom
krflol:python3
Open

python3 initial#9
krflol wants to merge 10 commits into
CREEATION:masterfrom
krflol:python3

Conversation

@krflol

@krflol krflol commented Jan 11, 2020

Copy link
Copy Markdown
Collaborator

Dataframe and pure python from sql implementation

krflol added 10 commits January 11, 2020 10:31
Dataframe and pure python from sql implementation
The data api and the public api are seperated. This commit includes the files to create (and hopefully eventually automate to some degree) database creation.
Added update function so that the api can be served via flask or something, and changes to the data can be reflected without reinitializing
@CREEATION

Copy link
Copy Markdown
Owner

Alright, quickly going through everything I think I'm understanding what you were building now.

This is an actual build implementation to build the JSON files, right? Because the data seems to be embedded in the Python files directly, which is exclusive to Python then. I wouldn't be able to easily make changes to the data without knowing and installing Python or other dependencies.

My initial thought was to establish a set of base JSON files to expand upon, not settle on any programming languages for the API. Then everyone can create a module for Python, JavaScript, you name it, all extending from the JSON files which everyone can easily edit/modify with just a text editor. This means you can also make your own SQL database using the JSON files.

That's why I was going for the string-arrays in JSON: they're meant to represent the data structure and dependencies, all having the same key names and overall structure. People can then figure out how to actually use the data and, hopefully, make pull requests with examples to help others build apps.

JSON can be included in any language or development environment without the need to install anything, because JSON is a fairly widespread standard for data exchange and easy to read (if you're not going for cryptic magic numbers, but strings). Maybe modules/plugins here and there are needed, I don't really know about C and other languages.

So basically, what I'm seeing here right now:

  • Base data (Brands, etc.) in Python files
  • SQL Database example (read/write/export to JSON)
  • Usage example for Python with SQL

Correct me if I'm wrong, please. 🐢

If I'm correct, this is a good start for an example of how to build upon the data and make a working Python application using SQL. But then we'd need to agree that we need the JSON files as a base, which your example should then reference, so everyone is working from the same set of data and can make structural requests for those if really needed.

For example, your comment about id currently being a string would not be the job of the JSON files, but the developer of the actual application. You can add your own unique id's programmatically using Python without problems and then create the SQL database from that data, without losing the base data.

Of course, this implies that you're just downloading the JSON files or otherwise referencing them (e.g. via a module) in your project. You could, to name a very simple example, keep them up to date by copy-pasting them over from this repo into your project and let your Python code handle how to make use of the given JSON data.

This is getting pretty long and I've lost the overview I think, so I'll leave it here for now. Please let me know what you think 🐢

@CREEATION CREEATION added examples question Further information is requested conversation General discussions and conversations data structure labels Jan 12, 2020
@krflol

krflol commented Jan 12, 2020

Copy link
Copy Markdown
Collaborator Author

Yes. I have one project for the creation of the files, and a separate project for a python implementation using the generated sql database

@krflol

krflol commented Jan 12, 2020

Copy link
Copy Markdown
Collaborator Author

And the data is not imbedded in python. The data project contains the logic to generate the sql database, which would really only have to be done once. After that, it would be very easy to maintain and export to json

@CREEATION

Copy link
Copy Markdown
Owner

And the data is not imbedded in python. The data project contains the logic to generate the sql database, which would really only have to be done once. After that, it would be very easy to maintain and export to json

But the data here is in the python file directly. Are you thinking to instead import from JSON there and the current code is just static placeholder data?

@krflol

krflol commented Jan 14, 2020

Copy link
Copy Markdown
Collaborator Author

That is the code that makes the sql database

@krflol

krflol commented Jan 14, 2020

Copy link
Copy Markdown
Collaborator Author

In other words, this stuff could be manually entered into a sql database with any number of tools, however, if we get a datamining solution, this is the start of the framework to automate the creation and updating of the sql database (which could then be export to json or whatever). I still need to tidy up the code and make it more elegant. It should look more like this psuedocode

fields = data.columns
for row in rows:
    brands.append(fields, row)

with something in brands like

def __init__(self, fields, row)
    for fields in fields:
        for item in row:
            self.setattribute(field, item)

That way it could pull data and generate much more efficiently, be automated, and it would be much more extensible, just havent gotten around to it.

@CREEATION

Copy link
Copy Markdown
Owner

Would you mind sharing contact information with me? It'd be easier to chat about this and present the results here. 🐢 You can find various ways to contact me on my website.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

conversation General discussions and conversations data structure examples question Further information is requested

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants