Sentdex Analysis

Linear Regression following Sentdex's tutorials

Hello. I am trying to do some machine learning on some bitcoin data, specifically linear regression. The full code is here, but in order to plot it on a graph, I want to use the values of y (which is the values of x in 14.5 days time, so price in 14.5 days time) where I use the old actual values of y followed by the new values of y which are the predictions. In order to do this I need to find the values of X which have values for y (the predictions) and the values for x which already have the price in 14.5 days time. I performed a shift on the data, meaning some Xs have values for Y in 14.5 days time and some don't.
Why 14.5 days? As the data set is 1450 days long and I did a 0.01 negative shift. Hopefully I communicated what I was trying to say alright.
import pandas as pd import math import numpy as np from sklearn import preprocessing, svm from sklearn.model_selection import cross_validate from sklearn.model_selection import train_test_split from sklearn.linear_model import LinearRegression from statistics import mean import matplotlib.pyplot as plt from matplotlib import style df = pd.read_csv("coinbaseUSD_1-min_data_2014-12-01_to_2019-01-09.csv") df['date'] = pd.to_datetime(df['Timestamp'],unit='s') print("calculating...") forecast_col = 'Weighted_Price' forecast_out = int(math.ceil(0.01*len(df))) #forecast_out = 20998 = 20998 minutes = 14.5 days df['label'] = df[forecast_col].shift(-forecast_out) df = df[['date', 'Weighted_Price', 'label']] df.dropna(inplace=True) X = np.array(df['Weighted_Price'], dtype = np.float64) y = np.array(df['label'], dtype=np.float64) X_lately = X[-forecast_out:] X = X[:-forecast_out:] def best_fit_line(X, y): m = (((mean(X) * mean(y)) - mean(X*y)) / ((mean(X) * mean(X)) - mean(X*X))) c = mean(y) - (m * (mean(X))) return m, c m, c = best_fit_line(X, y) print(m, c) regression_line = [(m*values) for values in X] plt.scatter(X, y) plt.plot(X, regression_line)

So what have I tried? The offender is this line here:
X_lately = X[-forecast_out:] X = X[:-forecast_out:]
That is what sentdex did in the video series, but I get the error: ValueError: operands could not be broadcast together with shapes (1871868,) (1892866,)
This doesn't work with:
m = (((mean(X) * mean(y)) - mean(X*y)) / ((mean(X) * mean(X)) - mean(X*X)))
due to this making the X and Ys different lengths? I'm not sure.
What am I doing wrong?
submitted by EnvironmentalPause5 to learnpython [link] [comments]

Long Term Bitcoin sentiment picking up steam

I track sentiment for stocks, politics, and some other things. One of the other things is bitcoin, thought I would share it with you:
Edit: Some people are questioning the accuracy here, apparently it's not clear enough. I made a temporary overlay chart. I'll work on adding overlay as an option for viewing for the public as well. Til then, here're some pictures:
Amplitude may not match up perfectly, but directions do pretty good.
submitted by sentdex to BitcoinMarkets [link] [comments]

Developing an open-source trading client, requests?

I am currently developing a trading client that will be completely open-source. For those of you who do not know me, I run a python programming / finance programming tutorial channel, as well as manage a bitcoin fund at, which is now expanding to offer more services.
I've been putting a lot of thought into creating a trading client for bitcoin, since most trading is going on still on the exchange's trading pages. I've used these before, and they are quite poor. Now, I do automated trading completely, but I want to push a trading client as well for people who are still forced to do trading on the exchange pages. I've already got quite a bit of the client created, and now I am working on adding features. I figured it'd be a good idea to see what people wanted instead of just using what I think is best.
One feature for sure that is needed on every exchange is stop-loss and trailing stop-loss, among quite a few other things. Bitfinex is one of the only exchanges to even bother with these sorts of things. I can also rather easily incorporate allowing people to trade based on simple strategies like SMA/EMA crossovers, MACD...etc...and even combinations of strategies. If it's based on TA, it can be done very simply, and other more complex strategies can be incorporated.
Right now, as far as I know the only comparable way to do this is via something like, where you have to sub to the site, and to a strategy. I believe I can do it for far cheaper, cutting out quite a bit of waste from that model, while also allowing people to create their own basic strategies without needing to understand any programming at all.
The client will be totally open-source on github once beta is pushed, and I will actually be covering it's entire creation via my YouTube channel as a tutorial, so stay tuned for that if you are interested in programming/python!
Sooooo suggestions on things to include? What do you wish exchanges had for you?
submitted by sentdex to BitcoinMarkets [link] [comments]

[P] Automatic Reddit Categorizer update (first version working!)

Yesterday I asked a few people about creating a program that can automatically categorize reddit submission into appropriate sub-reddit using Title text.
The first version is ready and it works!
A lot of people helped me yesterday and thanks to comments by deltasheep1 and olBaa I was able to create it. It is very bad code right now since I just learned python programming and my knowledge of ML is very limited (mainly sentdex videos), but I feel super happy that I was able to create this program.
Here is how I did it (please bear with me because I know I'm a total noob)
  1. First Run this Google Big Query
    SELECT subreddit, title FROM [fh-bigquery:reddit_posts.2017_03], [fh-bigquery:reddit_posts.2017_02], [fh-bigquery:reddit_posts.2017_01], [fh-bigquery:reddit_posts.2016_12], [fh-bigquery:reddit_posts.2016_11], [fh-bigquery:reddit_posts.2016_10], [fh-bigquery:reddit_posts.2016_09], WHERE subreddit IN ('programming', 'business', 'design', 'entertainment', 'science', 'security', 'worldnews', 'politics', 'mobile', 'startups', 'google', 'microsoft', 'bitcoin', 'facebook', 'amazon', 'movies', 'gadgets', 'technology', 'linux', 'gaming', 'apple', 'design', 'music' )
  2. Then export the results into a Table and export it again to a CSV file to your Google storage bucket
  3. Then install Google Data Labs and Create a notebook
  4. Then use the following program to run it!
I've still got a lot to learn and my next challenge is to create a small webpage so that people can enter the title in a web form and it shows the correct sub-reddit using it.
submitted by breakupnoob to learnmachinelearning [link] [comments]

Bitcoin BTCUSD sentiment analysis graph

Thought I'd share some of the sentiment analysis compared to price that I've done on Bitcoin over the past few months.
I do this mostly for stocks, but started doing it for bitcoin a bit ago out of fun/curiosity. I am pretty happy with how well it has turned out so far. Edit: Fixed the bad price glitches from BTC-e.
It also takes a bit to load. It's a lot of data. Expect to wait about a minute for it to load the actual chart.
To save loading if you'd like, here's just an image of the current graph:
submitted by sentdex to BitcoinMarkets [link] [comments]

Ethereum #3 on

I am posting here first because I was modded by u/ursium on ethereum while being redirected to post here. (I was going to post here anyway). Hopefully this shows up on ethereum. I am interested in other people's thoughts on this topic.
We are at the beginning of seeing a number of indices for Ethereum arise. Some of these feed from different data and aggregate types to provide reasonable insight from within the cryptosphere. Most here know as an example, or perhaps Sentdex - less well known.
Recently, brought Ethereum online, ranking it #3 (behind Litecoin and Bitcoin). If you don't already know check it out. If you know about it already and use it for investment analysis, than it's really quite easy just to think this post only belongs here. ranks each crypto network by weighting subcategories of criteria into a composite scores (between 0 - 100). Overall the Ethereum Network score looks pretty good, all things considered Ethereum is new. Nevertheless, one might consider a 47 score on "community" as a need for all of us to bolster social marketing. Is everyone here connected and reposting across your twitter, YouTube, FaceBook, G+, LinkedIn networks?
You might also look at "public interest". How is it that Ethereum has a lower Alexa score than Ripple? How do we improve that? What about "developer interest". Have you starred the cpp-ethereum repo on git hub already? What other channels not listed would help direct new people to the repo?
Maybe is more than a tool for financial planners. Maybe ETH DEV and the Ethereum community ought to look deeply at these and other indices for what they are all telling us, to grow Ethereum. Maybe folks here in the subs have good ideas not yet being expressed (BIG and little).
I would love to hear your thoughts and ideas.
submitted by PseudonymousChomsky to ethtrader [link] [comments]

1 step up from Basic Tutorials

Hello, my name is Harrison. I thought I would introduce myself here, I get a few mentions in here from time to time, but I thought I could also intro myself.
I graduated university about a year and a half ago, I own a few online businesses, about to get married <3, run a small-cap bitcoin fund and I do python tutorials on the side and have really enjoyed doing it a lot! I am nowhere near an expert overall in programming, and certainly do not know as much as many of you, but I try to do my part and cover non-beginner topics. It seemed like there exists a bunch of "intro" tutorials, both video and written, for various languages, but hardly anything for the next step. As I was trying to learn myself, I found that branching from beginner level to whatever is in between beginner and intermediate very difficult.
I cover mainly (all with python 2.7): Intros to machine learning/data analysis, finance, sentiment analysis, and Matplotlib/numpy, though there are many others in there, and not all videos are able to be on the homepage since I have over 400.
I come here for two reasons, 1 to just share my channel, but also I am curious if people have any requests or suggestions for the channel.
I have considered the conversion to 3, and doing py 3 versions of everything, but I am still not sold on py 3.... though I feel like it might just be a self-fulfilling prophesy since a lot of people still don't use py 3 simply because there isn't as much support...then there aren't people that port...and we have a circle.
I also just bought my 2nd pi, and plan to buy a few more. I want to do a nice simple super-computing tutorial with the pis, and also do some robotics.
Anyway, if you have suggestions/requests/comments, that'd be great. No need to tell me I'm a noob, I know, but you still can if you want. :)
The channel is:
Thx for readin!
submitted by sentdex to learnpython [link] [comments]

Bitcoin sentiment analysis + any interest in Geo sentiment?

You all like to talk and opine about bitcoin very much, I like to measure your feelings.
Bitcoin sentiment analysis chart:
Some of you might be familiar with it from before, but we've made some changes, including adding micropayments to view...and then removing them due to glitches as well as "micro" payments not being so micro after all...
The chart still takes a decent while to load, but we've revamped the code quite a bit to help with loading, but it still takes ~45 seconds.
Bitcoin is by far the largest topic Sentdex covers for sentiment analysis. The amount of data we collect on bitcoin is well over 10x our most popular talked about stock, which is Apple.
The goal is to use sentiment to predict price movements in Bitcoin / other stocks. Of course, many times price moves and then people get excited. This can be seen, but we've also been able to precede movements on many occasions.
Bitcoin is still a very new topic for us to cover. We've got a lot of data on it now, but we couldn't really go back in history as far for machine learning like we could with the rest of the stocks. Anyway, thought I'd share.
Curiously, I also wonder if there's any interest for people to see Geographic sentiment analysis for Bitcoin? I do geographic sentiment analysis here:
I've got things like Google and Facebook just for kicks. I'm also able to get popular words based on regions. Currently I only do this for the general sentiment chart, but I wonder too about doing this with Bitcoin. Thoughts? Might be pretty cool to see bitcoin sentiment by region.
submitted by sentdex to Bitcoin [link] [comments]

Thinking of trading Bitcoin? Live Trading Bitcoin +8% in 30 minutes - Is it worth it?

Hey fellow Bitcoin folks, thought I would share some of my findings with bitcoin here.
I run a youtube channel where I do a bunch of python/finance tutorials. I have been toying with Bitcoin for a bit, trading it for fun. I had a pretty good trading session today, which you can see here:
THAT said (shown :P) ... I would just like to share my thoughts with it here. I also come from a background of trading both by hand and by program in the stock market and other markets for profit. It is beginning to look like a commonality here that HFTing wont be an option... unless you were just recently with BTC China with no trade fees... which is epic (they no longer offer this... :( ).
Long term, it is probably best for Bitcoin to not favor HFTing, it is somewhat a problem in most stock markets.
Anyway just thought I'd share, have a nice day and best wishes for Bitcoin!
submitted by sentdex to Bitcoin [link] [comments]

Bitcoin Trading - Relative Strength Index (RSI) sentdex - YouTube Bitcoin - YouTube Bitcoin Protocol Explained - Timestamp Server / Global Ledger Bitcoin: Why Merchants Love Bitcoin and Cryptocurrencies

Bitcoin sentiment is pulled mainly from Reddit as the source of content, though most content that is digested contains links that lead off of Reddit. The algorithm reads both the articles linked to Reddit posts, as well as the comments. The algorithm does *not* read votes on threads. How Sentdex judges Sentiment. At its heart, Sentdex is a bot that reads the news like you or me. As Sentdex ... Bitcoin; London Stocks (LSE) Australian Securities (ASX) 7 Days. 30 Days. 6 months. 1 Year. All-Time. Overlay Sentiment. Volume time frame and overall sentiment time frame is determined by the current time frame of the company you are currently viewing. Change the graph's time frame to change this table's. Symbol Instrument Name ALL Volume of Mentions ALL Overall Sentiment Recent Sentiment ... "Sentiment analysis for Bitcoin is derived in a similar way that we derive sentiment for other topics. Sentdex uses a crawl bot (and a lot of bandwidth) constantly sifting through real-time information from news websites, forums, and other media outlets. Once data is acquired, natural language processing (NLP) techniques are used decipher underlying sentiment within text as a form of opinion ... Sentdex tracks over 600 company tickers, mainly consisting of S&P 500 companies. Want to make sure the company you want is included? Search our instrument selection. This table contains all of the tickers we track, including non-stocks like commodities and bitcoin. Search either for the ticker or the name of the instrument. Symbol Instrument Name; AGN: Allergan PLC: NYX: Euronext: NFLX ... GET NOW: sentdex - Live Bitcoin MINING Session +8% in 30 minutes - sentdex. ===== Live Bitcoin MINING Session +8% in 30 minutes is the procedure through which transactions are validated and contributed to the general public journal, called the block chain, and the means where brand-new bitcoin are launched.

[index] [7756] [32255] [48913] [6040] [28510] [20517] [24596] [11141] [36677] [36076]

Bitcoin Trading - Relative Strength Index (RSI)

I have been seeing a lot of comments that merchants wouldn't accept Bitcoin due to volatility, or anyone for that matter. This is sad to see, because it is absolutely not true. Accepting Bitcoin ... Welcome to part 8 of the Deep Learning with Python, Keras, and Tensorflow series. In this tutorial, we're going to work on using a recurrent neural network t... As Bitcoin, and other cryptocurrencies gain popularity, more and more people are curious about becoming a part of it. Many people start out curious about mining Bitcoins first, but then begin to ... As Bitcoin, and other cryptocurrencies gain popularity, more and more people are curious about becoming a part of it. Many people start out curious about mining Bitcoins first, but then begin to ... In this short video, the Bitcoin timestamp server / global ledger is discussed. As usual, however, we present ourselves with an ending challenge which is to be covered in the, you guessed it, next ...