LSTM Models, and Sentiment Analysis

I’ve implemented the basic LSTM model with Tensorflow’s Keras API, and tests look promising.

There is a major drawback — it’s dog shit slow. I mean “2 days for a full set of forecasts” slow, and that’s with optimizations before going distributed.

This is largely because Tensorflow is built for larger scale operations with much larger sets of data than I want to work with, with bigger hardware than I want to work with.

From looking around I think PyTorch shows some indicators of being more efficient for this use case, so, I’ll rename this to TF_LSTM and bring in a PT_LSTM as options. This updates the model portfolio to:

1. CatBoost (already implemented)
2. Prophet (already implemented)
3. (AUTO)ARIMA (already implemented)
4. TF_LSTM (already implemented)
5. PT_LSTM
6. XGBoost
7. 1D CNN
8. Ensemble

In terms of which is doing the best so far, other than LSTM, which seems to have hella good directional accuracy, they’re really all about the same so far until:

• I get to doing some more advanced data modeling, like adding in additional correlation metrics and introducing sentiment analysis.
• Hyperparameter tuning.

Right now Prophet is the only Argonaut wrapped model that has much hyperparameter tuning capability, because these have been drive-by implementations to just get Argonaut built. In Prophet’s case, much of the hyper-parameter tuning can be set in Argonaut to be done automationally by calculating ideal parameters based on the training data it’s training on for that forecast. I haven’t even started a pass on doing that for the other models just out of a lack of familiarity still with these other models I’m bringing in. There’s alot of math and alot of complex design and modeling in these that needs to be carefully considered for that, and I’ve only done that work for Prophet due to it being the model choice for previous builds of Tetsuo.

Regarding sentiment analysis, I think it’s a major pivot point. The online resources that would need consumed for this would merit a dedicated service that only serves the purpose of taking a date, a symbol, and returns a sentiment score.

This is doable, but, there are problems to work out due to the complexity it would introduce. The problem is that after obtaining a sentiment score for a symbol and a date, how to actually incorporate that sentiment score into forecast selection. The market is often wrong. The forecasts are often wrong. Usually not for the same reasons. Then there’s the delayed impact of sentiment-driving publications. On Tuesday an analyst publishes a bearish announcement about a symbol, but people don’t start reading it until Thursday. So containing sentiment to daily periodicity, which is what we’d need, may not be a sensible task. So it may just end up being a categorical indicator.

Maybe between the forecast metrics, forecasted performance, and sentiment score a “confidence weight” can be used to weight or boost forecasts in the sorted list when it’s evaluating the data, or maybe a high sentiment score would bump up the ranking of a good forecast with high quality metrics to give the tendency of winners to float to the top choices.

Or, maybe the sentiment analysis service could print a list of symbols it’s feeling bullish on, and we can only do forecasts of those. We’ll miss out on other forecasts though, and it’s not whole market analysis, so, I’ll have to put thought into whether that’s a good direction to steer it or not. It may be better to filter out stocks that have overwhelmingly bearish sentiment instead, and this would be more aligned to the project goal.

I’ll have to sleep on it for a bit. I don’t want to implement it until I have an idea of how I’ll use it.