- cross-posted to:
- technology@lemmy.world
- python@programming.dev
- cross-posted to:
- technology@lemmy.world
- python@programming.dev
Microsoft is bringing popular programming language Python to Excel. A public preview of the feature is available today, allowing Excel users to manipulate and analyze data from Python.
You won’t need to install any additional software or set up an add-on to access the functionality, as Python integration in Excel will be part of Excel’s built-in connectors and Power Query. Microsoft is also adding a new PY function that allows Python data to be exposed within the grid of an Excel spreadsheet. Through a partnership with Anaconda, an enterprise Python repository, popular Python libraries like pandas, statsmodels, and Matplotlib will be available in Excel.
You must log in or # to comment.
Python calculations run in the Microsoft Cloud,
Ah shit, so close.
I truly wonder why. I mean, others just package a python installation, but Microsoft wants to use the cloud. Very peculiar.
Running in the cloud is another reason to keep paying them every month in perpetuity, rather than just once. Helps keep revenues stable and indefinite.
It also allows them to completely gate the feature via tiers, like they do with other things in their environment. I’ve written about Power Platform since it is a pretty accessible tool for a lot of people. But it is also a shining example of Microsoft’s almost microtransaction-like enterprise vision of the future. Everything is great in the preview. While they collect usage data. Then they tuck the most useful and common functionality behind various paywalls, including per usage paywalls. They leave just enough in the base tier to draw people in and get them committed to the platform.
It will not surprise me in the least if basic features are removed and paywalled after the preview. It would not surprise me in the least if they repeat what they’ve already done and prevent users from using built-in python functions unless the user pays up.
But they can’t stop me from importing CSV data that I make with regular python. MS loves legacy features almost as much as they love money, that’s why windows 11 still has the fucking control panel.
Exactly why I hope it will flop and they decide it’s not worth the money. It’s disgusting how tech companies take something free and try to monetize it by hiding it under a veneer of “accessibility” and “integration”. One can only hope that information regarding FOSS will remain easily accessible and that the group this update is targeting is insignificant enough. I doubt any serious data scientist would opt to Excel + Python over just Python for big enough data sets, and the average Excel user, let’s face it, is unlikely to learn Python.
Probably to keep it proprietary. If they distributed Python with scientific packages it would be hackable, and they’d lose control.
I work as a cybersecurity consultant.
This is going to be excellent for business.
I had mostly the same reaction. The back of the box reads okay, but the potential is worrying. Even as a “cybersecurity news rubbernecker” (Seriously Risky Biz podcasts)
Finally. Another step to automate my work.
Excel formulas are way too limited and VBA is a nightmare to work with.
Will this actually automate your workflow?
It seems that this Python integration expects that the source data already exists within the Excel file and Python can essentially just be used to create either visuals or new tables within the same Excel file.
If that’s accurate, then this is intended exclusively for data analysis and not process automation. I don’t think this will allow people to enhance their existing Python based ETL jobs or create new ones because of this new integration. This does not seem to be a replacement/substitute for VBA or OfficeScripts. It also does not seem to be an alternative to Power Query. If anything, this seems to be most similar to Power Pivot.
After preview it will be a paid subscription additional to Office365, the same strategy as Office Copilot. I hate this policy of pay for a product then still paid for more functionalities… at the end of the year you will have paid a lot.
Microsoft says Python in Excel will be included in a Microsoft 365 subscription during the preview, but “some functionality will be restricted without a paid license” after the preview ends.
Yeah this is typical Microsoft looking at ways to force people up the price ladder. They did it with Power Platform in very obvious ways. They have completely gutted things like Power Apps and Power Automate by making almost all functions non-delegable… unless you are a paying a premium on top of a premium for costly dataverses in which case more than like 7 functions are magically delegable again. But then there are the pay-per-user/pay-per-use connections to access your own data, even if you host it yourself as an enterprise.
They should’ve been broken up in the late 90s.
But why does it need to run in the cloud?
💰💰💰 By sending every calculation to Microsoft servers they can log what your company is doing and sell that data to ad-agencies. Also it forces you into a subscription.
I’m not a fan of it either, but I’m not sure how else this could work seamlessly. How would you ensure that everyone you share your Excel file that utilizes Python has the expected Python setup on their machine? What if they have an older version of a library you used that breaks your script? What if they don’t have Python installed at all?
While this will only work on Windows desktop at first, Microsoft plans to roll this out to “other platforms” over time. Is there any other way for this integration to work for Excel for web, iPad, and/or Teams?
A python distribution could be bundled inside of excel.
That’s true. It wouldn’t solve the dependency issue though (eg - I’m using Pandas v1.5.3, you’re using Pandas v2.0.3) and I’m not sure how well it would work for some platforms like the web or Teams.
It could use a venv for each excel file.
I’m not sure exactly. Maybe it is one reason why they are pushing an online container. I guess then then it can have its own issues of dependency hell. At least with the container approach, everything either works or it is broken, and there is no having to verify which versions of Excel are installed to make the spreadsheet run.
I doubt the end user would have control over package versions, but I had not gotten around to playing with it yet.
A simple check? Then a prompt to download the expected version?
This isn’t rocket science. This is Microsoft forcing people to continually pay them.
Would something like this work for the web or Teams versions of Excel?
Regardless, I agree. The license and remote only execution are horrible.
Let’s see how they fuck this one up
Python with a freaking powerful GUI built in. Yes please!
This is the best summary I could come up with:
A public preview of the feature is available today, allowing Excel users to manipulate and analyze data from Python.
“You can manipulate and explore data in Excel using Python plots and libraries, and then use Excel’s formulas, charts and PivotTables to further refine your insights,” explains Stefan Kinnestrand, general manager of modern work at Microsoft.
Microsoft is also adding a new PY function that allows Python data to be exposed within the grid of an Excel spreadsheet.
Python calculations run in Microsoft’s Cloud, with the results returned into an Excel worksheet.
Excel users will be able to create formulas, PivotTables, and charts all based on Python data, with the ability to bring in charting libraries like Matplotlib and Seaborn for visualizations like heatmaps, violin plots, and swarm plots.
Python in Excel is rolling out today as a public preview for Microsoft 365 Insiders in the Beta Channel.
The original article contains 342 words, the summary contains 147 words. Saved 57%. I’m a bot and I’m open source!
As someone who is learning python, this is exactly what I was learning to code for. Are there any known resources or courses that specifically teach about python for excel or what you should learn t for this application? Libraries & dictionaries seem like a good start?
I’m guessing the features for reading csv will be important. So I would look at Automate the Boring Stuff with Python.
Cool, thanks for the suggestion!
There are some very good libraries for using Python with Excel. The creator of xlwings wrote a book: Python for Excel.
My favorite library is pandas, which can integrate with other libraries to edit Excel.
deleted by creator
Fantastic news. This will help me immensely!
Having it run locally would be even more help for my use cases. I often have lists of IP addresses in my sheets, and it would be helpful to ping them directly from Python code. But I can work around that with fping or nmap.
This integration won’t allow you to do that. Python will not run locally, but instead on Microsoft’s platform (likely Azure).
If you’re just reading some simple data from Excel, there are several ways of accomplishing this already. For example, Pandas has
read_excel()and there is also openpyxl. You could even use those tools to write the results back to Excel. Things get more complicated though if the Excel file is something more than just a simple list.And honestly, for data you should be using formats like .csv anyways. i really don’t see who this update is supposed to target. If you’re using Python, the data must be large enough to warrant its use.
Indeed
‘Tis better to .split() in the cloud than never to .split() at all
Python for Applications?
I’d like to see it. VBA is actually brilliant, you can make very very portable applications with intuitive user input and output interfaces (i.e. a cell range in Excel, and and version of tables and graphs you like as outputs)… But realistically who wants to use BASIC these days?
I usually say it in jest, but I’m not sure there is actually a better RAD (rapid application development) platform out there than Excel (with a bit of VBA as needed).That would be nice, but I think this is more “Power Pivot via Python”.
But why? Excel is a shit way to work with big amounts of data due to it’s own format’s complexity and bloated software. It’s welcome to implement python, but that’s not what holds it down. Opening a big csv would crash it on the same machine that loads it with a python IDE in seconds. It’s not made for this. It’s like, nice, but the volume of information you need to make it matter would break Excel in halves.
This feels like a really dated take to me. Leaving aside whether this was true in the past, in 2023, Excel is happy to open absolutely gargantuan files, and it’s quite speedy once it’s done so. You can even directly tie it to a database via ODBC if you want, and that works (albeit it obviously flattens the data out in the process, so goodbye foreign keys in any real sense). It also has tons of very easy-to-use data manipulation tools (pivot tables, tables in general, data extrapolation, graphs, etc.) that end up being wonderful complements to something like Python.
Could you write a Python program that would run faster than pure Excel and do the same thing? I mean, probably (although Excel’s core execution engine is honestly pretty freaking fast). But could you write it as quickly? Maybe, maybe not. And certainly someone who knows Excel well would have an easier time adding a little Python to patch up any issues than rewriting the whole thing from scratch.
tl;dr I think you’re not being accurate about contemporary Excel, and I separately suspect you’re not really the target audience here
Could I write a Python program that does the same thing as Excel but faster?
I don’t need to. It’s called pandas
I agree with all your points about Excel being capable. However, I’m struggling to think of examples where this newly announced Python integration within Excel would be helpful (with the exception of new/different visualizations) - especially for the reasons you stated about modern Excel.
Are there any use cases that you can think of where someone who knows Excel well would resort to “adding a little Python to patch up any issues”?
Maybe some sort of “team toolbox” of logical functions or something? I’ve seen some nightmare shops where big reports have a page of manipulations that get copied to new reports/projects every time, and each represents some sort of canned, core business logic.
I dunno. I cant imagine how the code storage will go
The code storage for Python is no different than regular Excel functions (eg -
VLOOKUP(),SUM(), etc.), meaning that it is stored within an Excel cell. The only differences are that Python code is run remotely vs Excel functions running locally and the location of Python’s code matters vs Excel’s functions are location agnostic (ie - Python code runs in cells located left-to-right, top-to-bottom but Excel’s functions can dynamically determine the calculation order/location).I’m not sure that this new Python integration changes much about this use case (except for another way to accomplish the same/similar tasks).
I hear you.
I don’t know how new Excel performes and I thought it’s the same as ten years ago - the version I’m trapped in. With people who obsessively try to drive it to the edge where it’s not responsive on average office PCs.
But if it works well with various big spreadsheets now, it’s a wonder, with how many new people start to tackle programming with Python. I obiviosly won’t write a script faster than normal operational speeds of software, it’s just some tables ended up that big and broken I could only open them like that. But that, I guess, is exclusion?
It’s just the issue of people using a microscope as a hammer when they need to break nuts.
So, assuming you’re still on Office 2010, you’re missing (off the top of my head, but I believe these were all Excel 2013 or later):
- Initial support for super-large spreadsheets, with accompanying perf improvements (this was maybe 2013?), and then lots more perf improvements in 2016 and 2019 as people started using those really big sheets;
- Flash Fill support, which is kind of like an AI (not in the ChatGPT-sense)-powered fill down (so e.g., nowadays, if I enter “Sept 6th” and then “October 4th”, then it’ll offer a completion of November 1, since that’s the first Wednesday of the month);
- Heavily improved Pivot Tables, including Slicers, Power Pivot, and Pivot Drill Down, which make exploring data a lot easier;
- Forecasting tools; and
- Much better Tables, making it a lot easier to write formulae entirely with named references rather than cell IDs
There’s more, but that’s what I could remember off the top of my head/could quickly verify so I was sure I wasn’t misspeaking. LibreOffice Calc unfortunately also lacks basically all of these features (and the lack of Tables in particular means that OpenOffice sheets still have a lot of
A2:A300garbage where Excel would instead just have e.g.SomeTable[Heading]. E.g., an actual formula from a sheet I currently maintain to track my team’s sprints:=XLOOKUP([@Verified],SprintMeta[Start],SprintMeta[Sprint Name],"Unknown",-1). Python’s easier to read here, but this is honestly doing a lot while being surprisingly readable (especially if you’re familiar withXLOOKUP, which is basically how you do keyed array access in Excel)
You have totally legitimate gripes about Excel; I’m not denying that. But I do think that you might be pleasantly surprised on newer versions.
Sounds like this won’t be the right tool for your use case
You are right, but it’d still be used by my company for me to cringe at that without a way to change it 😓
I agree! I’m not sure why you’re being downvoted either.
This new integration just allows you to do data analysis and data visualization of existing data within an Excel file via Python. The output of your Python scripts is limited to the Excel file. The Python environment itself is also limited as it runs on Microsoft’s platform and is controlled by Microsoft.
The (Excel) problem that people already using Python for data analysis/visualization is that they have to use Excel files. Reading/writing Excel files via Python can sometimes be tedious or limiting. Utilizing Python inside of Excel via this integration may help in some scenarios, but they won’t be able to use custom libraries built internally, control the Python environment (eg - must use specific version of Python or Python library, can’t utilize all Python libraries available on pip, etc.), connect to all necessary external data sources via Python, and utilize proper VCS tools like git.
The problem that people automating tasks via Python have is that there is no Python library nearly as capable of reading or manipulating Excel files as VBA is. This new Python integration does not change that.
The problem that Excel users have is that they want more advanced (or simple/easier) data analysis and data visualization capabilities. However, with Excel’s dynamic array formulas, LAMBDA formula, Power Query, and Power Pivot, Excel is becoming much more capable than it ever was. If those tools cannot meet your needs, you likely need to move to something like R, Python, or some other tool. Embedding Python into Excel like this integration does still limits Python with all of Excel’s current restraints (size, performance, etc.).
Because this may allow companies that are already using excel and not planning on changing to actually have a powerful programming language.
Does LibreOffice support python? because VB is cancer
Yes.
deleted by creator
Already does.
First FL Studio, now Excel haha
What about FL Studio?
FL Studio has python scripting in the latest update
That’s really cool! I know Python but always wanted to try making music in FL Studio. I doubt this will be easy though.
