Simple tool - Concatenating slides using FFmpeg. RabbitMQ(Message broker server) and Celery(Task queue). Neural Networks with backpropagation for XOR using one hidden layer įabric - streamlining the use of SSH for application deploymentĪnsible Quick Preview - Setting up web servers with Nginx, configure enviroments, and deploy an App įlask app with Apache WSGI on Ubuntu14/CentOS7. Removing Cloud Files - Rackspace API with curl and subprocessĬhecking if a process is running/hanging and stop/run a scheduled task on WindowsĪpache Spark 1.3 with PySpark (Spark Python API) Shellīottle 0.12.7 - Fast and simple WSGI-micro framework for small web-applications. Scheduled stopping and starting an AWS instanceĬloudera CDH5 - Scheduled stopping and starting services Uploading a big file to AWS S3 using boto module Simple tool - Google page ranking by keywords Python Unit Test - TDD using unittest.TestCase class Image processing with Python image library Pillow Python Network Programming IV - Asynchronous Request Handling : ThreadingMixIn and ForkingMixIn Python Network Programming III - Echo Server using socketserver network framework Python Network Programming II - Chat Server / Client Python Network Programming I - Basic Server / Client : B File Transfer Python Network Programming I - Basic Server / Client : A Basics REST API : Http Requests for Humans with Flask Web scraping with Selenium for checking domain availability Python HTTP Web Services - urllib, httplib2 MongoDB with PyMongo I - Installing MongoDB. Connecting to DB, create/drop table, and insert data into a table Priority queue and heap queue data structure Python Object Serialization - yaml and json Python Object Serialization - pickle and json Sets (union/intersection) and itertools - Jaccard coefficient and shingling to check plagiarismĬlasses and Instances (_init_, _call_, etc.)īits, bytes, bitstring, and constBitStream Strings - Escape Sequence, Raw String, and Slicingįormatting Strings - expressions and method calls Object Types - Numbers, Strings, and None Running Python Programs (os, sys, import) Then it calculates the tf-idf for each term found in an article. Then, we call fit_transform() which does a few things: first, it creates a dictionary of 'known' words based on the input text given to it. However, we used scikit-learn's built in stop word remove rather than NLTK's. TfidfVectorizer(tokenizer=tokenize, stop_words='english') Note that we pass the TfIdfVectorizer our own function that performs custom tokenization and stemming We converted the text to lowercase and removed punctuation. The Scicki-learn's sklearn.feature_extraction module can be used to extract features in a format supported by machine learning algorithms from datasets consisting of formats such as text and image.Īs we can see from the output, we iterate over the files in the Steinbeck collection. Str = 'all great and precious things are lonely.'įeature_names = tfidf.get_feature_names() Tfs = tfidf.fit_transform(token_dict.values()) Tfidf = TfidfVectorizer(tokenizer=tokenize, stop_words='english')
Token_dict = text.lower().translate(None, string.punctuation) The input files are from Steinbeck's Pearl ch1-6.įrom sklearn.feature_extraction.text import TfidfVectorizerįrom import PorterStemmerįor dirpath, dirs, files in os.walk(path):
#SUPER VECTORIZER UNINSTALL CODE#
Here is the code not much changed from the original: Document Similarity using NLTK and Scikit-Learn.
0 kommentar(er)
