In [1]:
%reload_ext autotime
import pandas as pd
from bertopic import BERTopic
from bertopic.representation import KeyBERTInspired
import plotly.io as pio
import plotly.express as px # Plotting
pio.renderers.default = 'notebook'
pd.options.plotting.backend = "plotly"
pd.set_option("display.max_columns", None)
pd.set_option("display.max_colwidth", 100)
df = pd.read_excel("givealittle_health.xlsx")
df.index = pd.to_datetime(df["start"])
df["text"] = df.title.str.cat(df[["pitch", "description", "use_of_funds"]].astype(str), sep=" ")
df
✔️ 14.2 s (2024-11-18T12:11:36/2024-11-18T12:11:50)
Out[1]:
| title | pitch | hero | uri | amountRaised | goal | timeLeft | action | actionUri | start | end | n_questions | location | n_donors | description | use_of_funds | whos_involved | moderated | n_updates | updates | Q&A | text | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| start | ||||||||||||||||||||||
| 2024-11-03 | Young mother of 3 recently diagnosed with aggressive breast cancer | Young mother of three fighting aggresive breast cancer. | https://static.givealittle.co.nz/assets/hero/e582c9fe-dee8-4746-91bf-b21d00ddf11e-320:False | https://givealittle.co.nz/cause/young-mother-of-3-recently-diagnosed | 3900.00 | 0.0 | 181 days to go | Donate | https://givealittle.co.nz/donate/cause/young-mother-of-3-recently-diagnosed | 3 Nov 2024 | 3 May 2025 | 0 | Pirimai, Napier | 38.0 | Imagine facing one of life's biggest battles while also caring for a young family.\nOur beloved ... | Travel costs for treatment.Financial help with household expenses.Cost to helping fix there vehi... | Created by_x000D_\nDebra Evans \n\n\n\n_x000D_\n Paying to a verified bank account of_x000... | Page Moderated | NaN | NaN | NaN | Young mother of 3 recently diagnosed with aggressive breast cancer Young mother of three fightin... |
| 2024-11-01 | Finlee Bremners Road to recovery | Helping our great friends navigate Finlee's long journey with kidney disease. | https://static.givealittle.co.nz/assets/hero/21b5840d-540f-49c1-bac0-b21800abc236-320:False | https://givealittle.co.nz/cause/finlee-bremners-road-to-recovery | 4500.00 | 0.0 | 177 days to go | Donate | https://givealittle.co.nz/donate/cause/finlee-bremners-road-to-recovery | 1 Nov 2024 | 29 Apr 2025 | 0 | Alexandra, Otago | 44.0 | On the 2nd October 2024, Kayla, Michael and Arlo welcomed a beautiful baby girl into the world, ... | All money received will go towards on going costs while Kayla and Michael are away from home wit... | Created by_x000D_\nKelsey McLean \n\n\n\n_x000D_\n Paying to a verified bank account of_x0... | Page Moderated | NaN | NaN | NaN | Finlee Bremners Road to recovery Helping our great friends navigate Finlee's long journey with k... |
| 2024-10-31 | Paula Matafahi's Grade 4 Brain Cancer Fight - We need your help | This page has been set up to help Paula access unfunded drugs to offer hope and relieve his symp... | https://static.givealittle.co.nz/assets/hero/bd8b1741-7704-41df-bc4d-b21a0105b9dc-320:False | https://givealittle.co.nz/cause/paulas-stage-4-brain-cancer-fight-please-help | 47522.15 | 55000.0 | 27 days to go | Donate | https://givealittle.co.nz/donate/cause/paulas-stage-4-brain-cancer-fight-please-help | 31 Oct 2024 | 30 Nov 2024 | 0 | Manawatu / Whanganui | 628.0 | Paula is an extraordinary 18 year old young man whose spirit shines brightly even in adversity.\... | Providing the potentially life extending but unfunded drug (Avastin) and to help with the cost o... | Created by_x000D_\nSheree Turia \n\n\n\n_x000D_\n Paying to a verified bank account of_x00... | Page Moderated | 2.0 | Nearly there!\n3 November 2024\nWe are just about at $47,000!!! Thank you all again for your gen... | NaN | Paula Matafahi's Grade 4 Brain Cancer Fight - We need your help This page has been set up to hel... |
| 2024-10-31 | Our Sweet boy Jairo | Please help Jairo get the testing and screening that he needs | https://static.givealittle.co.nz/assets/hero/2f49c6b1-267a-4422-8fe8-b21a00cc4779-320:False | https://givealittle.co.nz/cause/our-sweet-boy-jairo | 925.00 | 1000.0 | 19 days to go | Donate | https://givealittle.co.nz/donate/cause/our-sweet-boy-jairo | 31 Oct 2024 | 22 Nov 2024 | 0 | Palmerston North, Manawatu-Wanganui | 9.0 | My 8 week old baby Jairo has had chronic collic and constipation since he was born. I believe he... | It will be used for the consult and further testing as well as petrol to get to Wellington and back | Created by, and paying to a verified bank account of,_x000D_\nDanni Gillett on behalf of My... | Page Moderated | 1.0 | Follow up appointments\n2 November 2024\nJust updating the goal for follow up appointments x\nSh... | NaN | Our Sweet boy Jairo Please help Jairo get the testing and screening that he needs My 8 week old ... |
| 2024-10-30 | Em’s MS Journey | Stemcell replacement for Em’s MS Journey. Clínica Ruiz In Mexico. | https://static.givealittle.co.nz/assets/hero/68655bdf-a7bf-4e0c-a9ab-b2190184b42a-320:False | https://givealittle.co.nz/cause/ems-ms-journey | 9640.00 | 90000.0 | 178 days to go | Donate | https://givealittle.co.nz/donate/cause/ems-ms-journey | 30 Oct 2024 | 30 Apr 2025 | 0 | Dunedin, Otago | 136.0 | My daughter is 25 and has recently been diagnosed with Multiple Sclerosis. Em was working fullti... | Flights to Mexico and operation for Stemcell Replacement/ Chemo/ medication. The operation invol... | Created by, and paying to a verified bank account of,_x000D_\nDiane Stanley on behalf of Em... | Page Moderated | NaN | NaN | NaN | Em’s MS Journey Stemcell replacement for Em’s MS Journey. Clínica Ruiz In Mexico. My daughter is... |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 2016-05-02 | Amber and Daniel were twins born on 24th and 25th December 2015. They were born at 23 weeks gest... | Raising money for the little McSweeney family who have spent the last 5 months in hospital! | https://static.givealittle.co.nz/assets/hero/a07c15e8-a593-48bb-a652-a5fa009e35b9-320:False | https://givealittle.co.nz/cause/hartleytwinsmicropremmies | 4834.00 | 0.0 | closed | Read more | https://givealittle.co.nz/cause/hartleytwinsmicropremmies | 2 May 2016 | 6 Jun 2016 | 0 | Canterbury | 122.0 | PLEASE READ THE UPDATE TO THIS POST UNDER UPDATE TAB. We need you all to know how much of an imp... | NaN | Created by, and paying to a verified bank account of,_x000D_\n _x000D_\n Elise Hartl... | Page Moderated | 1.0 | Attention all followers of our story\n31 May 2016\nHello, my name is Elise Hartley and I am the ... | NaN | Amber and Daniel were twins born on 24th and 25th December 2015. They were born at 23 weeks gest... |
| 2016-05-01 | Tania Whanau, known by most as Tarnz, is a strong, 40 year old woman who was diagnosed with brea... | Since February '16 Tarnz has undergone two operations; one removing 34 lymph nodes from her righ... | https://static.givealittle.co.nz/assets/hero/652566e9-6f90-4632-9b3f-a5f900e9af2c-320:False | https://givealittle.co.nz/cause/fightbreastcancerlikeabitch | 1550.00 | 0.0 | closed | Read more | https://givealittle.co.nz/cause/fightbreastcancerlikeabitch | 1 May 2016 | 22 Jul 2016 | 0 | Hawke's Bay | 34.0 | Tania Whanau, known by most as Tarnz, is a strong, 40 year old woman who was diagnosed with Grad... | NaN | Created by_x000D_\n _x000D_\n Nancy Easthope_x000D_\n \n\n\n\n\n_x000D_\n ... | Page Moderated | NaN | NaN | NaN | Tania Whanau, known by most as Tarnz, is a strong, 40 year old woman who was diagnosed with brea... |
| 2016-05-01 | Cardiology Fundraiser | Fundraising for patients & families affected by Heart Disease. | https://static.givealittle.co.nz/assets/hero/bfefeb94-f041-478f-80a0-a5f900c0dc47-320:False | https://givealittle.co.nz/cause/fund4aheart | 915.00 | 0.0 | closed | Read more | https://givealittle.co.nz/cause/fund4aheart | 1 May 2016 | 15 Aug 2016 | 0 | Waikato | 9.0 | We aim to provide clients with the most up-to-date education around their Heart Disease diagnosi... | NaN | Created by, and paying to a verified bank account of,_x000D_\n _x000D_\n Cardiology ... | Page Moderated | NaN | NaN | NaN | Cardiology Fundraiser Fundraising for patients & families affected by Heart Disease. We aim to p... |
| 2016-04-30 | Help for Julia, while she is caring for Ian Robertson | Please support me so I'll be able to continue caring for my dear friend Ian Roberston. He suffer... | https://static.givealittle.co.nz/assets/hero/531d73ca-22d8-423b-93be-a5f700a2c8ed-320:False | https://givealittle.co.nz/cause/help4julia | 2645.00 | 0.0 | closed | Read more | https://givealittle.co.nz/cause/help4julia | 30 Apr 2016 | 30 May 2016 | 0 | Canterbury | 31.0 | I'm caring for my dear Kiwi friend Ian Robertson, who suffers from terminal lung cancer. It's ge... | NaN | Created by, and paying to a verified bank account of,_x000D_\n _x000D_\n Julia Schlu... | Page Moderated | NaN | NaN | NaN | Help for Julia, while she is caring for Ian Robertson Please support me so I'll be able to conti... |
| 2016-04-29 | TRUS Biopsy Machine for Taranaki | The men of Taranaki need this TRUS Biopsy Machine to diagnose Prostate Cancer. Reduce waiting t... | https://static.givealittle.co.nz/assets/hero/c9169845-63b5-4c1e-bb50-a5fc00e5abd0-320:False | https://givealittle.co.nz/cause/prostatebiopsymachinenewplymouth | 1910.00 | 20000.0 | closed | Read more | https://givealittle.co.nz/cause/prostatebiopsymachinenewplymouth | 29 Apr 2016 | 31 Aug 2016 | 0 | Taranaki | 15.0 | Currently the TRUS (Trans Rectal Ultrasound) machine for diagnosing prostate cancer is only avai... | NaN | Created by, and paying to a verified bank account of,_x000D_\n _x000D_\n Rotary Club... | Page Moderated | NaN | NaN | NaN | TRUS Biopsy Machine for Taranaki The men of Taranaki need this TRUS Biopsy Machine to diagnose P... |
10418 rows × 22 columns
In [2]:
df.amountRaised.resample("1YE").sum().plot.bar()
✔️ 640 ms (2024-11-18T12:11:50/2024-11-18T12:11:51)
In [3]:
df.resample("ME").size().plot()
✔️ 124 ms (2024-11-18T12:11:51/2024-11-18T12:11:51)
In [14]:
keywords = ["covid", "cancer", "leukemia", "heart", "stroke", "kidney", "bowel"]
for word in keywords:
df[word] = df.text.str.lower().str.contains(rf"\b{word}\b")
df
✔️ 1.93 s (2024-11-18T12:23:07/2024-11-18T12:23:09)
Out[14]:
| title | pitch | hero | uri | amountRaised | goal | timeLeft | action | actionUri | start | end | n_questions | location | n_donors | description | use_of_funds | whos_involved | moderated | n_updates | updates | Q&A | text | covid | cancer | leukemia | heart | stroke | kidney | bowel | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| start | |||||||||||||||||||||||||||||
| 2024-11-03 | Young mother of 3 recently diagnosed with aggressive breast cancer | Young mother of three fighting aggresive breast cancer. | https://static.givealittle.co.nz/assets/hero/e582c9fe-dee8-4746-91bf-b21d00ddf11e-320:False | https://givealittle.co.nz/cause/young-mother-of-3-recently-diagnosed | 3900.00 | 0.0 | 181 days to go | Donate | https://givealittle.co.nz/donate/cause/young-mother-of-3-recently-diagnosed | 3 Nov 2024 | 3 May 2025 | 0 | Pirimai, Napier | 38.0 | Imagine facing one of life's biggest battles while also caring for a young family.\nOur beloved ... | Travel costs for treatment.Financial help with household expenses.Cost to helping fix there vehi... | Created by_x000D_\nDebra Evans \n\n\n\n_x000D_\n Paying to a verified bank account of_x000... | Page Moderated | NaN | NaN | NaN | Young mother of 3 recently diagnosed with aggressive breast cancer Young mother of three fightin... | False | True | False | False | False | False | False |
| 2024-11-01 | Finlee Bremners Road to recovery | Helping our great friends navigate Finlee's long journey with kidney disease. | https://static.givealittle.co.nz/assets/hero/21b5840d-540f-49c1-bac0-b21800abc236-320:False | https://givealittle.co.nz/cause/finlee-bremners-road-to-recovery | 4500.00 | 0.0 | 177 days to go | Donate | https://givealittle.co.nz/donate/cause/finlee-bremners-road-to-recovery | 1 Nov 2024 | 29 Apr 2025 | 0 | Alexandra, Otago | 44.0 | On the 2nd October 2024, Kayla, Michael and Arlo welcomed a beautiful baby girl into the world, ... | All money received will go towards on going costs while Kayla and Michael are away from home wit... | Created by_x000D_\nKelsey McLean \n\n\n\n_x000D_\n Paying to a verified bank account of_x0... | Page Moderated | NaN | NaN | NaN | Finlee Bremners Road to recovery Helping our great friends navigate Finlee's long journey with k... | False | False | False | False | False | True | False |
| 2024-10-31 | Paula Matafahi's Grade 4 Brain Cancer Fight - We need your help | This page has been set up to help Paula access unfunded drugs to offer hope and relieve his symp... | https://static.givealittle.co.nz/assets/hero/bd8b1741-7704-41df-bc4d-b21a0105b9dc-320:False | https://givealittle.co.nz/cause/paulas-stage-4-brain-cancer-fight-please-help | 47522.15 | 55000.0 | 27 days to go | Donate | https://givealittle.co.nz/donate/cause/paulas-stage-4-brain-cancer-fight-please-help | 31 Oct 2024 | 30 Nov 2024 | 0 | Manawatu / Whanganui | 628.0 | Paula is an extraordinary 18 year old young man whose spirit shines brightly even in adversity.\... | Providing the potentially life extending but unfunded drug (Avastin) and to help with the cost o... | Created by_x000D_\nSheree Turia \n\n\n\n_x000D_\n Paying to a verified bank account of_x00... | Page Moderated | 2.0 | Nearly there!\n3 November 2024\nWe are just about at $47,000!!! Thank you all again for your gen... | NaN | Paula Matafahi's Grade 4 Brain Cancer Fight - We need your help This page has been set up to hel... | False | True | False | False | False | False | False |
| 2024-10-31 | Our Sweet boy Jairo | Please help Jairo get the testing and screening that he needs | https://static.givealittle.co.nz/assets/hero/2f49c6b1-267a-4422-8fe8-b21a00cc4779-320:False | https://givealittle.co.nz/cause/our-sweet-boy-jairo | 925.00 | 1000.0 | 19 days to go | Donate | https://givealittle.co.nz/donate/cause/our-sweet-boy-jairo | 31 Oct 2024 | 22 Nov 2024 | 0 | Palmerston North, Manawatu-Wanganui | 9.0 | My 8 week old baby Jairo has had chronic collic and constipation since he was born. I believe he... | It will be used for the consult and further testing as well as petrol to get to Wellington and back | Created by, and paying to a verified bank account of,_x000D_\nDanni Gillett on behalf of My... | Page Moderated | 1.0 | Follow up appointments\n2 November 2024\nJust updating the goal for follow up appointments x\nSh... | NaN | Our Sweet boy Jairo Please help Jairo get the testing and screening that he needs My 8 week old ... | False | False | False | False | False | False | False |
| 2024-10-30 | Em’s MS Journey | Stemcell replacement for Em’s MS Journey. Clínica Ruiz In Mexico. | https://static.givealittle.co.nz/assets/hero/68655bdf-a7bf-4e0c-a9ab-b2190184b42a-320:False | https://givealittle.co.nz/cause/ems-ms-journey | 9640.00 | 90000.0 | 178 days to go | Donate | https://givealittle.co.nz/donate/cause/ems-ms-journey | 30 Oct 2024 | 30 Apr 2025 | 0 | Dunedin, Otago | 136.0 | My daughter is 25 and has recently been diagnosed with Multiple Sclerosis. Em was working fullti... | Flights to Mexico and operation for Stemcell Replacement/ Chemo/ medication. The operation invol... | Created by, and paying to a verified bank account of,_x000D_\nDiane Stanley on behalf of Em... | Page Moderated | NaN | NaN | NaN | Em’s MS Journey Stemcell replacement for Em’s MS Journey. Clínica Ruiz In Mexico. My daughter is... | False | False | False | False | False | False | False |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 2016-05-02 | Amber and Daniel were twins born on 24th and 25th December 2015. They were born at 23 weeks gest... | Raising money for the little McSweeney family who have spent the last 5 months in hospital! | https://static.givealittle.co.nz/assets/hero/a07c15e8-a593-48bb-a652-a5fa009e35b9-320:False | https://givealittle.co.nz/cause/hartleytwinsmicropremmies | 4834.00 | 0.0 | closed | Read more | https://givealittle.co.nz/cause/hartleytwinsmicropremmies | 2 May 2016 | 6 Jun 2016 | 0 | Canterbury | 122.0 | PLEASE READ THE UPDATE TO THIS POST UNDER UPDATE TAB. We need you all to know how much of an imp... | NaN | Created by, and paying to a verified bank account of,_x000D_\n _x000D_\n Elise Hartl... | Page Moderated | 1.0 | Attention all followers of our story\n31 May 2016\nHello, my name is Elise Hartley and I am the ... | NaN | Amber and Daniel were twins born on 24th and 25th December 2015. They were born at 23 weeks gest... | False | False | False | False | False | False | False |
| 2016-05-01 | Tania Whanau, known by most as Tarnz, is a strong, 40 year old woman who was diagnosed with brea... | Since February '16 Tarnz has undergone two operations; one removing 34 lymph nodes from her righ... | https://static.givealittle.co.nz/assets/hero/652566e9-6f90-4632-9b3f-a5f900e9af2c-320:False | https://givealittle.co.nz/cause/fightbreastcancerlikeabitch | 1550.00 | 0.0 | closed | Read more | https://givealittle.co.nz/cause/fightbreastcancerlikeabitch | 1 May 2016 | 22 Jul 2016 | 0 | Hawke's Bay | 34.0 | Tania Whanau, known by most as Tarnz, is a strong, 40 year old woman who was diagnosed with Grad... | NaN | Created by_x000D_\n _x000D_\n Nancy Easthope_x000D_\n \n\n\n\n\n_x000D_\n ... | Page Moderated | NaN | NaN | NaN | Tania Whanau, known by most as Tarnz, is a strong, 40 year old woman who was diagnosed with brea... | False | True | False | False | False | False | False |
| 2016-05-01 | Cardiology Fundraiser | Fundraising for patients & families affected by Heart Disease. | https://static.givealittle.co.nz/assets/hero/bfefeb94-f041-478f-80a0-a5f900c0dc47-320:False | https://givealittle.co.nz/cause/fund4aheart | 915.00 | 0.0 | closed | Read more | https://givealittle.co.nz/cause/fund4aheart | 1 May 2016 | 15 Aug 2016 | 0 | Waikato | 9.0 | We aim to provide clients with the most up-to-date education around their Heart Disease diagnosi... | NaN | Created by, and paying to a verified bank account of,_x000D_\n _x000D_\n Cardiology ... | Page Moderated | NaN | NaN | NaN | Cardiology Fundraiser Fundraising for patients & families affected by Heart Disease. We aim to p... | False | False | False | True | False | False | False |
| 2016-04-30 | Help for Julia, while she is caring for Ian Robertson | Please support me so I'll be able to continue caring for my dear friend Ian Roberston. He suffer... | https://static.givealittle.co.nz/assets/hero/531d73ca-22d8-423b-93be-a5f700a2c8ed-320:False | https://givealittle.co.nz/cause/help4julia | 2645.00 | 0.0 | closed | Read more | https://givealittle.co.nz/cause/help4julia | 30 Apr 2016 | 30 May 2016 | 0 | Canterbury | 31.0 | I'm caring for my dear Kiwi friend Ian Robertson, who suffers from terminal lung cancer. It's ge... | NaN | Created by, and paying to a verified bank account of,_x000D_\n _x000D_\n Julia Schlu... | Page Moderated | NaN | NaN | NaN | Help for Julia, while she is caring for Ian Robertson Please support me so I'll be able to conti... | False | True | False | True | False | False | False |
| 2016-04-29 | TRUS Biopsy Machine for Taranaki | The men of Taranaki need this TRUS Biopsy Machine to diagnose Prostate Cancer. Reduce waiting t... | https://static.givealittle.co.nz/assets/hero/c9169845-63b5-4c1e-bb50-a5fc00e5abd0-320:False | https://givealittle.co.nz/cause/prostatebiopsymachinenewplymouth | 1910.00 | 20000.0 | closed | Read more | https://givealittle.co.nz/cause/prostatebiopsymachinenewplymouth | 29 Apr 2016 | 31 Aug 2016 | 0 | Taranaki | 15.0 | Currently the TRUS (Trans Rectal Ultrasound) machine for diagnosing prostate cancer is only avai... | NaN | Created by, and paying to a verified bank account of,_x000D_\n _x000D_\n Rotary Club... | Page Moderated | NaN | NaN | NaN | TRUS Biopsy Machine for Taranaki The men of Taranaki need this TRUS Biopsy Machine to diagnose P... | False | True | False | False | False | False | False |
10418 rows × 29 columns
In [16]:
df.resample("1M").cancer.sum().describe()
✔️ 46.8 ms (2024-11-18T12:23:15/2024-11-18T12:23:15)
Out[16]:
count 104.000000 mean 37.576923 std 9.600457 min 1.000000 25% 32.000000 50% 38.000000 75% 44.000000 max 58.000000 Name: cancer, dtype: float64
In [17]:
df.resample("1M")[keywords].sum().plot(height=800)
✔️ 97 ms (2024-11-18T12:23:17/2024-11-18T12:23:17)
In [7]:
topic_model = BERTopic(representation_model=KeyBERTInspired(), min_topic_size=10, verbose=True)
topic_model.fit(df.text)
topic_model.get_topic_info()
✔️ 37.2 s (2024-11-05T14:42:22/2024-11-05T14:42:59)
2024-11-05 14:42:22,114 - BERTopic - Embedding - Transforming documents to embeddings.
Batches: 0%| | 0/326 [00:00<?, ?it/s]
2024-11-05 14:42:37,106 - BERTopic - Embedding - Completed ✓ 2024-11-05 14:42:37,107 - BERTopic - Dimensionality - Fitting the dimensionality reduction algorithm 2024-11-05 14:42:53,056 - BERTopic - Dimensionality - Completed ✓ 2024-11-05 14:42:53,058 - BERTopic - Cluster - Start clustering the reduced embeddings 2024-11-05 14:42:53,586 - BERTopic - Cluster - Completed ✓ 2024-11-05 14:42:53,590 - BERTopic - Representation - Extracting topics from clusters using representation models. 2024-11-05 14:42:58,260 - BERTopic - Representation - Completed ✓
Out[7]:
| Topic | Count | Name | Representation | Representative_Docs | |
|---|---|---|---|---|---|
| 0 | -1 | 5203 | -1_cancer_care_diagnosed_family | [cancer, care, diagnosed, family, hospital, support, treatment, life, medical, surgery] | [Emergency travel to support grieving and sick parent. On Tuesday 28th July my mother had a stro... |
| 1 | 0 | 429 | 0_rehabilitation_recovery_rehab_injury | [rehabilitation, recovery, rehab, injury, stroke, injuries, support, spinal, family, care] | [Nick was involved in a truck accident and has a serious back injury with a long road ahead to r... |
| 2 | 1 | 303 | 1_cancer_chemotherapy_diagnosed_treatments | [cancer, chemotherapy, diagnosed, treatments, chemo, treatment, her, she, daughter, sister] | [Help Vanessa Beat Cancer Vanessa has aggressive stage 3 Inflammatory Breast Cancer. She needs a... |
| 3 | 2 | 289 | 2_charity_charities_depression_suicide | [charity, charities, depression, suicide, health, wellbeing, counselling, support, supporting, m... | [Help me so I can help others. I'm just a kiwi girl, coming forward about mental health. While y... |
| 4 | 3 | 248 | 3_family_support_aunty_accommodation | [family, support, aunty, accommodation, whanau, hospital, care, neonatal, babies, funds] | [Pete Waipouri & Whanau - Help us bring Pete home Pete is in critical condition in a hospital in... |
| ... | ... | ... | ... | ... | ... |
| 82 | 81 | 12 | 81_barre_paralysis_syndrome_disease | [barre, paralysis, syndrome, disease, rehabilitation, guillain, autoimmune, recovery, guillainba... | [Help Alex Reilly On His Road To Recovery Help Alex Reilly on his road to recovery from Guillain... |
| 83 | 82 | 11 | 82_hospital_maddie_daughter_blood | [hospital, maddie, daughter, blood, breathing, surgery, ordeal, ventilator, lung, emily] | [Emilys Story <3 Fundraising for Waikato NICU to help the amazing team provide further resources... |
| 84 | 83 | 11 | 83_transplant_heart_family_hearty | [transplant, heart, family, hearty, donations, hospital, support, aorta, children, healthy] | [Stage two, operation number two We have set up baby Carlo a give a little page in hope we can r... |
| 85 | 84 | 10 | 84_sanitation_water_cholera_orphanage | [sanitation, water, cholera, orphanage, donating, beyondwater, uganda, toilets, kenya, lesotho] | [Clean Water for Cambodia! Drinking from dirty water sources is the number one cause of illness ... |
| 86 | 85 | 10 | 85_treatment_paralysed_wheelchair_recovery | [treatment, paralysed, wheelchair, recovery, spinal, therapy, myelitis, gabby, atrophy, specialist] | [Save Heath from becoming completly trapped in his body Spinraza is the only hope we have of sa... |
87 rows × 5 columns
In [8]:
topic_model.visualize_topics()
✔️ 2.36 s (2024-11-05T14:42:59/2024-11-05T14:43:01)
In [9]:
topics_over_time = topic_model.topics_over_time(df.text, df.index, nr_bins=10)
topics_over_time
✔️ 26.8 s (2024-11-05T14:43:01/2024-11-05T14:43:28)
10it [00:26, 2.67s/it]
Out[9]:
| Topic | Words | Frequency | Timestamp | |
|---|---|---|---|---|
| 0 | -1 | diagnosed, cancer, treatment, surgery, hospital | 449 | 2016-04-25 21:21:36 |
| 1 | 0 | injury, recovery, injuries, hospital, rehabilitation | 45 | 2016-04-25 21:21:36 |
| 2 | 1 | cancer, chemotherapy, lump, diagnosed, treatments | 11 | 2016-04-25 21:21:36 |
| 3 | 2 | depression, suicide, health, grief, charity | 32 | 2016-04-25 21:21:36 |
| 4 | 3 | whanau, family, neonatal, auckland, babies | 16 | 2016-04-25 21:21:36 |
| ... | ... | ... | ... | ... |
| 768 | 76 | wheelchair, mobility, funds, chair, stroke | 2 | 2023-12-28 00:00:00 |
| 769 | 77 | donating, funds, surgeries, treatment, surgery | 2 | 2023-12-28 00:00:00 |
| 770 | 79 | kelly, kellys, her, rylee, cancer | 3 | 2023-12-28 00:00:00 |
| 771 | 81 | parkinsons, hamish, barre, disease, syndrome | 1 | 2023-12-28 00:00:00 |
| 772 | 82 | courtney, courtneys, birth, lung, hospital | 1 | 2023-12-28 00:00:00 |
773 rows × 4 columns
In [10]:
topic_model.visualize_topics_over_time(topics_over_time, top_n_topics=20, height=600)
✔️ 50.4 ms (2024-11-05T14:43:28/2024-11-05T14:43:28)
In [11]:
df = df[(df.timeLeft == "closed") & (df.goal > 0)]
df["Target completion %"] = df.amountRaised / df.goal * 100
display(df["Target completion %"].describe())
df.sort_values("Target completion %")
✔️ 29.5 ms (2024-11-05T14:43:28/2024-11-05T14:43:28)
count 4118.000000 mean 66.797576 std 799.319922 min 0.000000 25% 6.100000 50% 23.331667 75% 61.820455 max 44000.000000 Name: Target completion %, dtype: float64
Out[11]:
| title | pitch | hero | uri | amountRaised | goal | timeLeft | action | actionUri | start | end | n_questions | location | n_donors | description | use_of_funds | whos_involved | moderated | n_updates | updates | Q&A | text | covid | cancer | leukemia | heart | stroke | kidney | Target completion % | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| start | |||||||||||||||||||||||||||||
| 2024-10-13 | Flights needed NZ-London to help 83 year old mum through a mastectomy | Raising funds to send a caring daughter to London to look after her 83 year old mum following a ... | https://static.givealittle.co.nz/assets/hero/03a34b2c-1401-4460-8ca3-b20800b6f41e-320:False | https://givealittle.co.nz/cause/flights-needed-nz-london-to-help-83-year-old-mum | 0.0 | 3340.0 | closed | Read more | https://givealittle.co.nz/cause/flights-needed-nz-london-to-help-83-year-old-mum | 13 Oct 2024 | 16 Oct 2024 | 0 | NaN | NaN | My mother has breast cancer and at 83 is having a mastectomy on 24th October. She lives alone an... | Just the flights | Created by, and paying to a verified bank account of,_x000D_\n _x000D_\n Sian Jones | Page Moderated | NaN | NaN | NaN | Flights needed NZ-London to help 83 year old mum through a mastectomy Raising funds to send a ca... | 0 | 1 | 0 | 0 | 0 | 0 | 0.00000 |
| 2024-06-01 | Me and you against the world | Help keep my bro alive ❤️ | https://static.givealittle.co.nz/assets/hero/11e8353e-d3b1-4e38-983f-b18200aeb589-320:False | https://givealittle.co.nz/cause/me-and-you-against-the-world | 0.0 | 5000.0 | closed | Read more | https://givealittle.co.nz/cause/me-and-you-against-the-world | 1 Jun 2024 | 24 Jun 2024 | 0 | Bay of Plenty | NaN | Hi my name is Jess, my younger brother Corey (28 years old) was diagnosed with a 9cm pituitary g... | Medical expenses - to get an appointment with RestoreMe Clinic and subsequently CBD oil (prescri... | Created by_x000D_\nJessica Kelly \n\n\n\n_x000D_\n Paying to a verified bank account of_x0... | Page Moderated | NaN | NaN | NaN | Me and you against the world Help keep my bro alive ❤️ Hi my name is Jess, my younger brother Co... | 0 | 0 | 0 | 1 | 0 | 0 | 0.00000 |
| 2017-03-18 | Filling removal to reverse multiple sclerosis | Silver filling removal to cure m.s. caused by mercury poisoning | https://static.givealittle.co.nz/assets/hero/97b5a6a5-6fb9-432b-9bb0-a73a0110af59-320:False | https://givealittle.co.nz/cause/silveramalgalmscauseautoimmunedisorders | 4.0 | 250000.0 | closed | Read more | https://givealittle.co.nz/cause/silveramalgalmscauseautoimmunedisorders | 18 Mar 2017 | 20 Jun 2017 | 0 | Northland | 3.0 | Old filling removal is dangerous and requires a dentist that really knows how to remove them saf... | Silver filling removal, 2 teeth now need to be ripped as as root canal will harbour bacteria and... | Created by, and paying to a verified bank account of,_x000D_\n _x000D_\n Sandy Howell | Page Moderated | 1.0 | 28th april\n3 May 2017\nHad two teeth removed as they required root canals which are a death sen... | NaN | Filling removal to reverse multiple sclerosis Silver filling removal to cure m.s. caused by merc... | 0 | 0 | 0 | 1 | 0 | 0 | 0.00160 |
| 2023-05-19 | Charlies life threatening lung complication. | This page is set up to help support my family during this time of extreme hardship we are experi... | https://static.givealittle.co.nz/assets/hero/bf6e7924-c106-4b1e-b4bf-b0070056a009-320:False | https://givealittle.co.nz/cause/charlies-life-threatening-lung-complication | 7.0 | 100000.0 | closed | Read more | https://givealittle.co.nz/cause/charlies-life-threatening-lung-complication | 19 May 2023 | 10 Oct 2023 | 0 | Auckland | 3.0 | Please help our family as our lives have changed due to our sons right lung not functioning prop... | A vehicle, accomadation in a warm home. 2 sons under 3 + new born twins, funds will support us u... | Created by, and paying to a verified bank account of,_x000D_\nRobert Shane Williams | Page Moderated | NaN | NaN | NaN | Charlies life threatening lung complication. This page is set up to help support my family durin... | 0 | 0 | 0 | 0 | 0 | 0 | 0.00700 |
| 2019-03-05 | Help My Father | My Father is diagnosed with Adult CHD. His position is really critical, please help me so that i... | https://static.givealittle.co.nz/assets/hero/761d7080-ddd4-42e0-9dea-aa07014cb211-320:False | https://givealittle.co.nz/cause/help-my-father | 35.0 | 400000.0 | closed | Read more | https://givealittle.co.nz/cause/help-my-father | 5 Mar 2019 | 30 Apr 2019 | 0 | Auckland | 3.0 | Hello everyone,\nMy name is Manisha i am a single mother from India currently living in New Zeal... | For my father treatment | Created by, and paying to a verified bank account of,_x000D_\nmanisha mittal on behalf of M... | Page Moderated | NaN | NaN | NaN | Help My Father My Father is diagnosed with Adult CHD. His position is really critical, please he... | 0 | 0 | 0 | 2 | 0 | 0 | 0.00875 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 2017-01-20 | Cancer (Langhan's Cell Histiocytosis) | Raising money for treatment and recovery of being paralyzed and going through chemotherapy and h... | https://static.givealittle.co.nz/assets/hero/7ca74a99-5881-48a7-a7e4-a6fe0052b1bb-320:False | https://givealittle.co.nz/cause/shaunmurphy | 1160.0 | 10.0 | closed | Read more | https://givealittle.co.nz/cause/shaunmurphy | 20 Jan 2017 | 8 Jul 2017 | 0 | Auckland | 27.0 | Hi, I'm Shaun Murphy and I'm 25 years old.\nIf you don't already know my story, 2 years ago I st... | Rehabilitation treatment and any other costs for chemotherapy and physio | Created by, and paying to a verified bank account of,_x000D_\n _x000D_\n Shaun murphy | Page Moderated | NaN | NaN | NaN | Cancer (Langhan's Cell Histiocytosis) Raising money for treatment and recovery of being paralyze... | 0 | 6 | 0 | 0 | 0 | 0 | 11600.00000 |
| 2017-03-19 | We need to raise funds for Cibele's surgery | Please help raise funds to pay for a tumour removal surgery. Any donation will help. Thank you. | https://static.givealittle.co.nz/assets/hero/f2446f46-19a2-418b-b6fb-a73b014c6d84-320:False | https://givealittle.co.nz/cause/help4cibele | 2482.0 | 20.0 | closed | Read more | https://givealittle.co.nz/cause/help4cibele | 19 Mar 2017 | 1 May 2017 | 0 | Auckland | 36.0 | Cibele has a tumour in her stomach and desperately needs $20.000 to pay for a surgery to remove ... | To pay for a surgery to remove a tumour in her stomach. | Created by_x000D_\n _x000D_\n Carmosina Braga Trancoso_x000D_\n \n\n\n\n\n\n\n_... | Page Moderated | NaN | NaN | NaN | We need to raise funds for Cibele's surgery Please help raise funds to pay for a tumour removal ... | 0 | 0 | 0 | 0 | 0 | 0 | 12410.00000 |
| 2017-03-22 | To raise money to fight for Justice for Baby Jai curtis Blake preventable Still BIrth | Angela and Scott are seeking support with legal fees into the Still Birth of then unborn child t... | https://static.givealittle.co.nz/assets/hero/14ca9772-ba50-4f2d-91b6-a73e01280edc-320:False | https://givealittle.co.nz/cause/supportbabyjai | 277.0 | 2.0 | closed | Read more | https://givealittle.co.nz/cause/supportbabyjai | 22 Mar 2017 | 30 Apr 2017 | 0 | Canterbury | 16.0 | Angela lost her baby in utero at 37 weeks full term after a complicated pregnancy and lots of he... | funds are to fight the justice system for other women preventing this happening to other expecta... | Created by_x000D_\n _x000D_\n April Ritchie_x000D_\n \n\n\n\n\n\n\n_x000D_\n ... | Page Moderated | NaN | NaN | NaN | To raise money to fight for Justice for Baby Jai curtis Blake preventable Still BIrth Angela and... | 0 | 0 | 0 | 0 | 0 | 0 | 13850.00000 |
| 2017-07-18 | helping barry after a serious hit n run accident | our friend was in a serious hit and run and will be in hospital and after care for some time and... | https://static.givealittle.co.nz/assets/hero/2bfd0a24-b2ab-412f-b8f8-a7b400cce458-320:False | https://givealittle.co.nz/cause/barrystollery | 1450.0 | 10.0 | closed | Read more | https://givealittle.co.nz/cause/barrystollery | 18 Jul 2017 | 30 Sep 2017 | 0 | Bay of Plenty | 18.0 | One of our local motorcyclists was severly injured in a hit and run this week in tauranga he is ... | help barry to get on his feet and back to some kind of normality ideally he will need to save fo... | Created by_x000D_\n _x000D_\n Briar Morreel_x000D_\n \n\n\n\n\n\n\n_x000D_\n ... | Page Moderated | NaN | NaN | NaN | helping barry after a serious hit n run accident our friend was in a serious hit and run and wil... | 0 | 0 | 0 | 0 | 0 | 0 | 14500.00000 |
| 2022-02-24 | Help the Mason family | William's upcoming surgery/recovery | https://static.givealittle.co.nz/assets/hero/10960d59-c892-40b4-8121-ae4a0142f5fa-320:False | https://givealittle.co.nz/cause/help-the-mason-family | 2200.0 | 5.0 | closed | Read more | https://givealittle.co.nz/cause/help-the-mason-family | 24 Feb 2022 | 25 Jun 2022 | 0 | Upper Hutt, Wellington | 38.0 | Hi my name is William and I'm 38 years old and I had a massive Heart attack on the 2nd of Februa... | Everything. Travel to appointments in lower hutt and Wellington hospital ,Food, rent, petrol, ki... | Created by, and paying to a verified bank account of,_x000D_\n _x000D_\n William Mason | Page Moderated | 4.0 | Thanx all\n12 March 2022\nThanx all your support so far , I can't thank you enough. Have got 2 m... | NaN | Help the Mason family William's upcoming surgery/recovery Hi my name is William and I'm 38 years... | 0 | 0 | 0 | 2 | 0 | 0 | 44000.00000 |
4118 rows × 29 columns
In [14]:
# Take the last part of the location string (most coarse location)
df.location = df.location.str.split(", ").str[-1]
top_10 = df.location.value_counts().head(10)
top_10
✔️ 9.87 ms (2024-11-05T14:45:10/2024-11-05T14:45:10)
Out[14]:
location Auckland 1232 Canterbury 496 Wellington 409 Waikato 331 Nationwide 231 Bay of Plenty 186 International 149 Northland 140 Otago 140 Manawatu / Whanganui 124 Name: count, dtype: int64
In [16]:
df[df.location.isin(top_10.index)].groupby("location")["Target completion %"].mean().sort_values(ascending=False).plot.bar(title="Mean target completion % by location")
✔️ 34.5 ms (2024-11-05T14:45:33/2024-11-05T14:45:33)
In [17]:
df[df.location.isin(top_10.index)].groupby("location")["Target completion %"].median().sort_values(ascending=False).plot.bar(title="Median target completion % by location")
✔️ 34.5 ms (2024-11-05T14:45:33/2024-11-05T14:45:33)
In [18]:
fig = px.box(df[df.location.isin(top_10.index)], x="location", y="Target completion %", title="Target completion % by location")
fig.update_yaxes(range=[0, 200])
fig.show()
✔️ 37.1 ms (2024-11-05T14:45:33/2024-11-05T14:45:33)
In [19]:
px.scatter(df, x="n_donors", y="Target completion %", trendline="ols")
✔️ 467 ms (2024-11-05T14:45:34/2024-11-05T14:45:34)
In [20]:
px.scatter(df[(df.n_donors <= 200) & (df["Target completion %"] <= 110)], x="n_donors", y="Target completion %", trendline="ols")
✔️ 38.1 ms (2024-11-05T14:45:34/2024-11-05T14:45:34)
In [21]:
df["duration"] = (pd.to_datetime(df.end) - pd.to_datetime(df.start)).dt.days
px.scatter(df, x="duration", y="Target completion %", trendline="ols")
✔️ 50.4 ms (2024-11-05T14:45:34/2024-11-05T14:45:34)
In [22]:
px.scatter(df[(df.duration <= 100) & (df["Target completion %"] < 200)], x="duration", y="Target completion %", trendline="ols")
✔️ 41.9 ms (2024-11-05T14:45:35/2024-11-05T14:45:35)
In [25]:
import statsmodels.formula.api as smf
nona = df.dropna(subset=["location", "n_donors", "duration"]).rename(columns={"Target completion %": "Target_completion"})
nona["Cancer"] = nona.text.str.contains("Cancer", case=False).astype(int)
nona["Surgery"] = nona.text.str.contains("Surgery", case=False).astype(int)
nona["Auckland"] = nona.location.str.contains("Auckland").astype(int)
nona["Wellington"] = nona.location.str.contains("Wellington").astype(int)
smf.ols(formula="Target_completion ~ Cancer + Surgery + Auckland + Wellington + n_donors + duration", data=nona).fit().summary()
✔️ 128 ms (2024-11-05T14:46:34/2024-11-05T14:46:34)
Out[25]:
| Dep. Variable: | Target_completion | R-squared: | 0.002 |
|---|---|---|---|
| Model: | OLS | Adj. R-squared: | 0.001 |
| Method: | Least Squares | F-statistic: | 1.371 |
| Date: | Tue, 05 Nov 2024 | Prob (F-statistic): | 0.222 |
| Time: | 14:46:34 | Log-Likelihood: | -33248. |
| No. Observations: | 4103 | AIC: | 6.651e+04 |
| Df Residuals: | 4096 | BIC: | 6.655e+04 |
| Df Model: | 6 | ||
| Covariance Type: | nonrobust |
| coef | std err | t | P>|t| | [0.025 | 0.975] | |
|---|---|---|---|---|---|---|
| Intercept | 66.1470 | 22.843 | 2.896 | 0.004 | 21.363 | 110.931 |
| Cancer | -10.0779 | 27.272 | -0.370 | 0.712 | -63.545 | 43.389 |
| Surgery | 16.1703 | 26.832 | 0.603 | 0.547 | -36.434 | 68.775 |
| Auckland | 3.4612 | 27.974 | 0.124 | 0.902 | -51.382 | 58.305 |
| Wellington | 101.4573 | 42.719 | 2.375 | 0.018 | 17.704 | 185.211 |
| n_donors | 0.0379 | 0.051 | 0.741 | 0.459 | -0.062 | 0.138 |
| duration | -0.1184 | 0.090 | -1.322 | 0.186 | -0.294 | 0.057 |
| Omnibus: | 12445.848 | Durbin-Watson: | 2.003 |
|---|---|---|---|
| Prob(Omnibus): | 0.000 | Jarque-Bera (JB): | 879170249.105 |
| Skew: | 44.410 | Prob(JB): | 0.00 |
| Kurtosis: | 2268.990 | Cond. No. | 970. |
Notes:
[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.