Group Abstract Group Abstract

Message Boards Message Boards

Bayesian linear regression in Mathematica

POSTED BY: Sjoerd Smit
19 Replies
Posted 4 years ago

ARMA or ARIMA would be common choices for this type of time series data, I was looking at a similar dataset related to river bed depth and tried numerous approaches (e.g. RNN, an empirical physics based predictor using MCMC to fit parameters). If you're interested in a Bayesian approach to time series predictions then Bayesian structured time series is one possibility. The Inferring causal impact using Bayesian structural time-series models paper looks like a lot of work to implement but there is a R package (bsts) available (I haven't tried using it though) discussed in this article Sorry ARIMA but I'm going Bayesian. Without trying it I don't know whether it would OK through ExternalEvaluate or maybe through RLink.

POSTED BY: Steve Walker

Dear Sjoerd,

I have a time series of water temperature between 2019-2020 with a 15-minute interval for SKOKOMISH river, in Mason County, United States. Please see the enclosed file.

Do you have any idea for modeling this data for the very shortly (15 min or hourly) prediction process? Look forward to hearing your response.

Regards, Ghorbani

Attachments:
POSTED BY: M.A. Ghorbani

Dear Ghorbani,

It's difficult to say what would be a good model for this. There's a clear trend that can probably be fitted by a deterministic model. I'm not quite sure what you would do with the residuals after that. Presumably some model that works well for weather phenomena in general since I expect the weather to be the main driver for those temperature fluctuations. Maybe an ARMAProcess or something like that? I'm no expert on weather or climate, so I don't know what would be appropriate, to be honest.

POSTED BY: Sjoerd Smit

Attached is a notebook showing the MultivariateTDistribution for the distribution of the regression coefficients that I get out of LinearModelFit and the ones obtained from BayesianLinearRegression. I am leaning to the theory that there is nothing wrong with either my notebook or your code and that the problem is in my failure to understand Bayesian Linear Regression fully. But any enlightenment you could provide would be appreciated.

Attachments:
POSTED BY: Seth Chandler
POSTED BY: Sjoerd Smit
POSTED BY: Seth Chandler
POSTED BY: Sjoerd Smit

Hello Sjoerd, I am excited to find your post telling that you have developed "Bayesian Linear Regression," which I was finding. Most attractive to me is that you don't have to specify prior distribution. My question is

  1. what is the 4 distributions associated with keys "Posterior" or "Prior"?
  2. can you add Weibull distribution, if not currently included?

Thank you in advance. - Toshi

POSTED BY: Toshiyuki Meshii

Hi Toshi,

The default prior distribution used is very non-informative, so normally you don't need to worry about it much except when you do want to use an informative prior. The model uses a conjugate prior (because that allows closed-form solutions) that is explained on the two Wikipedia pages I linked:

For an example of specifying the prior, please look at "PriorParameters" section (under Options) on the documentation page of my resource function:

https://resources.wolframcloud.com/FunctionRepository/resources/BayesianLinearRegression

As for including a Weibull distrbution: is there a conjugate prior for that? If not, then the current function cannot use it. The current function uses these specific distributions because there exists conjugate priors for them. I don't think there's many other choices for which nice closed-form solutions like this.

Best regards, Sjoerd

POSTED BY: Sjoerd Smit
POSTED BY: Toshiyuki Meshii
POSTED BY: Sjoerd Smit
POSTED BY: Sjoerd Smit

Thanks for such a prompt reply, I look forward to giving it a try now.

Posted 6 years ago
POSTED BY: Steve Walker

Yes, I saw those errors as well. I don't actually know where they come from; probably some part of the internal MCMC sampler I'm using. It may be a fairly harmless thing, but I don't really know to be honest. The thing with MCMC and related methods is that things hardly ever work right out of the box without tweaking some options and settings. If you let the sampler run for a while (a couple of minutes or so) it should return a result.

POSTED BY: Sjoerd Smit
Posted 6 years ago

Hi Sjoerd

This is all really interesting material, thanks for posting it. Working through the BayesianInference examples, I ran into a problem with "Logistic classification with multi dimensional input and multiple classes" examples, the defineInferenceProblem evaluation shows $Failed for "GeneratingDistribution". I tried to figure out what might be amiss but without success - any suggestions?

I am using the package updated in July and the corresponding examples notebook running on version 12.0

-Steve

POSTED BY: Steve Walker

Hi Steve,

Yes, I can see what you ran into. I will admit that that particular example isn't worked out all that well. I more or less worked that out for myself and plonked it in for future reference. I assume you're referring to the fact that the tooltip of the inferenceObject shows the "GeneratingDistribution" key as $Failed? From what I can tell, that's just a bug in the code that generates the tooltip: if you evaluate obj["GeneratingDistribution"] it should still return the ProbabilityDistribution that was used to in the construction of the object. I'll try and patch it out in the next push to master.

It's continuing work in progress, though I my time and attention span to work on it is not super consistent. Glad to hear that you're giving it a try, though!

POSTED BY: Sjoerd Smit

enter image description here - Congratulations! This post is now featured in our Staff Pick column as distinguished by a badge on your profile of a Featured Contributor! Thank you, keep it coming, and consider contributing your work to the The Notebook Archive!

POSTED BY: EDITORIAL BOARD
Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard