getQuote.yahoo not working

[zlfang00]

getQuote not works any more:

getQuote("AAPL")
Error in .yahooSession(TRUE) : unable to get yahoo crumb

or windows:

getQuote("AAPL")
Error in open.connection(con, "rb") : HTTP error 401.

1, calling from Virginia/USA
2, has access to Yahoo Finance website quotes.

[joshuaulrich]

I can replicate this and it looks like it's the same thing as what happened to the python yfinance library.

[eddelbuettel]

yfinance works fine (for now?) as an alternate for quotes. I use the current PyPI version.

Demo

$ Rscript yfinanceDemo.r 
 Symbol                Time   Open    High    Low  Close   Volume Pct_change   Change
    SPY 2025-05-07 12:30:59 560.15 561.965 558.38 561.28 19012610   0.443816 2.480042
    QQQ 2025-05-07 12:31:01 482.11 484.000 479.42 482.13 16141147   0.149561 0.720001
$ 

Code

suppressMessages({
    library(reticulate)
})

symbols <- c("SPY", "QQQ")
yf <- import("yfinance")

get_data <- function(symbol) {
    dat <- yf$Ticker(symbol)$info
    data.frame(Symbol=symbol,
               Time=as.POSIXct(dat$regularMarketTime),
               Open=dat$regularMarketOpen,
               High=dat$regularMarketDayHigh,
               Low=dat$regularMarketDayLow,
               Close=dat$regularMarketPrice,
               Volume=dat$regularMarketVolume,
               Pct_change=dat$regularMarketChangePercent,
               Change=dat$regularMarketChange)
}

print(do.call(rbind, lapply(symbols, get_data)), row.names=FALSE)

It is the cookie / crumb business that blows out getQuote, as it (likely) does for Stocks Extension. I rewrote a few tasks here (feeding Redis, feeding an EOD database) using yfinance via reticulate.

[ethanbsmith]

i have an idea. if you could share the values for Sys.info()["sysname"] from R and the user-agent from you browser, ill cook up a POSSIBLE patch

[eddelbuettel]

sysname (no surprise here):
Linux

user-agent per server logs and a detecting browser page:
Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36

[ethanbsmith]

if u want to try https://github.com/ethanbsmith/quantmod/tree/438_getQuote_user_agent_fix
its just an idea

[eddelbuettel]

edd@rob:/tmp/r/quantmod(438_getQuote_user_agent_fix)$ install.r
* installing *source* package found in current working directory ...
* installing *source* package ‘quantmod’ ...
** this is package ‘quantmod’ version ‘0.4.27.1’
** using staged installation
** R
** demo
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (quantmod)
edd@rob:/tmp/r/quantmod(438_getQuote_user_agent_fix)$ Rscript -e 'quantmod::getQuote("SPY")'
Registered S3 method overwritten by 'quantmod':
  method            from
  as.zoo.data.frame zoo 
Error in .yahooSession(TRUE) : unable to get yahoo crumb
Calls: <Anonymous> ... do.call -> getQuote.yahoo -> .yahooSession -> .yahooSession
Execution halted
edd@rob:/tmp/r/quantmod(438_getQuote_user_agent_fix)$ 

No change in observed behaviour. yfinance and its Ticker continue to work.

[eddelbuettel]

c55d41e works here too:

edd@rob:~/git/quantmod(438-getquote-yahoo-too-many-requests)$ install.r
* installing *source* package found in current working directory ...
* installing *source* package ‘quantmod’ ...
** this is package ‘quantmod’ version ‘0.4.27.1’
** using staged installation
** libs
Warning: no source files found
** R
** demo
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (quantmod)
edd@rob:~/git/quantmod(438-getquote-yahoo-too-many-requests)$ Rscript -e 'quantmod::getQuote("SPY")'
Registered S3 method overwritten by 'quantmod':
  method            from
  as.zoo.data.frame zoo 
             Trade Time   Last  Change % Change   Open   High    Low   Volume
SPY 2025-05-07 16:00:00 561.15 2.35004 0.420551 560.15 563.82 556.04 47687752
edd@rob:~/git/quantmod(438-getquote-yahoo-too-many-requests)$ 

[joshuaulrich]

Note that this won't work on Windows (yet) because Windows uses Schannel instead of OpenSSL by default. I'll start working on figuring out the set of ciphers for Windows tomorrow, unless someone beats me to it (I hope).

I also have no idea about MacOS and would appreciate anyone's help with testing/fixing this issue on that OS.

[ethanbsmith]

@zlfang00 im trying to figure out why your windows test failed, but mine works. can you tell me more about your windows setup? for starts, can you share your sessionInfo() output

[joshuaulrich]

I get Error in .yahooSession(TRUE) : unable to get yahoo crumb on my Windows machine. I'm working on a fix. I have an hour before I have to leave for my kids' hockey game.

[zlfang00]

@zlfang00 im trying to figure out why your windows test failed, but mine works. can you tell me more about your windows setup? for starts, can you share your sessionInfo() output

here you are:

> sessionInfo()
R version 4.0.5 (2021-03-31)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19045)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252    LC_MONETARY=English_United States.1252 LC_NUMERIC=C                           LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] quantmod_0.4.18 TTR_0.24.2      xts_0.12.1      zoo_1.8-9      

loaded via a namespace (and not attached):
 [1] httr_1.4.2      compiler_4.0.5  R6_2.5.0        tools_4.0.5     curl_4.3.1      tinytex_0.44    grid_4.0.5      jsonlite_1.7.2  xfun_0.30       lattice_0.20-41

[zlfang00]

am I missing something? still have the error after installed the fixed version

> install.packages(".", repos=NULL, type="source")
Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
* installing *source* package ‘quantmod’ ...
** using staged installation
** R
** demo
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (quantmod)
> require(quantmod)
Loading required package: quantmod
Loading required package: xts
Loading required package: zoo

Attaching package: ‘zoo’

The following objects are masked from ‘package:base’:

    as.Date, as.Date.numeric

Loading required package: TTR
Registered S3 method overwritten by 'quantmod':
  method            from
  as.zoo.data.frame zoo
> getQuote("SPY")
Error in .yahooSession(TRUE) : unable to get yahoo crumb

[joshuaulrich]

am I missing something? still have the error after installed the fixed version

@zlfang00 yes, I said in an earlier comment after pushing the fix, "Note that this won't work on Windows (yet) because Windows uses Schannel instead of OpenSSL by default."

And I have since discovered that you cannot set ciphers using Schannel with the libcurl that comes with R's curl package.