Home Blog Page 3931

Kamala Harris’s DNC speech must take a web page from Joe Biden’s e book

0


President Joe Biden guess his reelection bid on the concept American voters in November would care about the destiny of American democracy. The typical knowledge now is that Biden was flawed and that Vice President Kamala Harris will not be solely stepping away from his technique however doing so for good causes.

Many Democratic political sorts right now imagine that “democracy” is a message that’s each drained and summary, one thing that preoccupies elites however not the important thing voting demographics. They cite information displaying ”democracy” on the backside of the checklist of voter considerations within the 2024 election, with points like inflation, immigration, and abortion inserting nicely forward of what Joe Biden handled as concern primary.

It seems Harris has taken the traditional knowledge to coronary heart. Reporting in Axios and the Washington Publish exhibits that Harris makes use of the phrase “democracy” far lower than Biden did in each promoting and stump speeches — mentioning it solely 12 instances in her first 9 marketing campaign rallies.

Reasonably than casting the Trump-Vance ticket as a menace to democracy, the Harris-Walz marketing campaign prefers to explain them as “bizarre” males aiming to curtail People’ freedom. The language of freedom has suffused the Democratic Nationwide Conference thus far, and can virtually definitely be a centerpiece of Harris’s deal with on Thursday evening.

But I feel the traditional knowledge is flawed in practically each respect. Harris mustn’t abandon democracy as a marketing campaign concern, however moderately double down on it.

Democracy is a extra highly effective concern than polling would possibly counsel: It doubtless swung the 2022 midterms Democrats’ means, and a few analysis suggests it stays potent in 2024. Democracy messaging is particularly persuasive with reasonable Trump-skeptical Republicans, an necessary voting bloc in swing states across the nation. Given its confirmed observe document of success with a vital demographic, democracy must be a big a part of Harris’s message.

Integrating democracy doesn’t require ditching the clearly efficient freedom messaging: Harris can fairly simply spend extra time highlighting assaults on democratic freedoms, speaking about “the liberty to vote” as one among a number of important liberties Trump and Mission 2025 plan to assault. That’s what a number of the Democratic pollsters who helped develop the celebration’s “freedom” message again in 2022 at the moment urge her to do.

“Saying ‘we’re making an attempt to guard your freedom to vote’ and ‘we’re making an attempt to guard your freedom to have a household whenever you wish to’… these two issues collectively body that story in a means that makes individuals care about democracy,” Jen Fernandez Ancona, vice chairman of the progressive technique group Manner To Win, informed Vox.

If Harris listens to the traditional knowledge as a substitute, she’ll be leaving a robust marketing campaign concern — and doubtlessly some essential votes — on the desk.

The case that democracy issues in American politics

Sarah Longwell is a longtime Republican strategist who, since 2016, has made it her life’s work to determine the way to get her fellow conservatives to ditch the Republican Celebration. As the pinnacle of a gaggle referred to as Republican Voters Towards Trump, she has performed “lots of and lots of” of focus teams with persuadable Republican moderates — a scientific try to determine how they is perhaps satisfied to ditch the Trumpified GOP.

When she talks to former Trump voters, they are saying the identical factor over and over: that his assault on democracy pushed them out of the celebration.

“I take heed to why voters who voted for Trump now refuse to take action, and the No. 1 motive is that January 6 was a pink line for them. They had been out after that,” Longwell says.

The 2022 midterms confirmed that these voters mattered — that these voters aren’t simply necessary, however doubtlessly decisive in vital states.

We bear in mind 2022 as a Democratic victory as a result of the celebration managed to make beneficial properties within the Senate whereas struggling minimal losses within the Home — defying the historic rule that the celebration in command of the White Home loses within the midterms. However this wasn’t the results of a blue wave: Republicans acquired a number of million extra votes in Home races nationally than Democrats. What occurred as a substitute is that Democrats excelled in a handful of key races, making historical past because of this.

So what occurred in these key races? The Dobbs ruling was a significant a part of the story. However so too was democracy.

A number of separate information analyses, performed by sources starting from Stanford political scientists to the Democratic information agency Catalist, tried to estimate a “MAGA penalty”: how a lot Republicans who parroted Trump’s “large lie” about 2020 suffered on the poll field relative to different Republicans. On the entire, these estimates converge someplace within the neighborhood of a two- to three-point penalty for MAGA candidates, with some estimates going as excessive as 5 or 6.

With out this impact, Democrats doubtless wouldn’t management the Senate: Georgia Sen. Raphael Warnock beat his opponent, Trump darling Herschel Walker, by 2.8 share factors. Furthermore, extremists would virtually definitely maintain key statewide positions: Kari Lake, the Republican candidate for governor in Arizona who made the large lie central to her message, misplaced by lower than a share level.

After the election, I interviewed a number of Democratic candidates and strategists who fought and received races towards election deniers. They informed me that placing democracy first was a significant element of their victory.

“Voters are simply uninterested in the bullshit. They realize it’s nonsense, that there’s no proof behind the large lie or any of that stuff, they usually rejected it,” Adrian Fontes, Arizona’s secretary of state who was first elected in 2022, informed me on the time.

Democratic strategists are absolutely conscious of what occurred final election, however not all of them are satisfied that it’ll work the identical means in 2024.

“[Democracy messaging] labored extraordinarily nicely with the oldsters we wanted within the midterms, which was these suburban college-educated of us who voted each time and don’t like the present route of the Republican Celebration. [But] these of us are already with us,” says Lanae Erickson, a senior vice chairman on the center-left Third Manner suppose tank.

Which may not be true. Anat Shenker-Osorio, the pinnacle of the progressive messaging outfit ASO Communications, just lately performed a randomized experiment testing the impact of adverts that includes January 6. She discovered that the adverts “did an honest job” swinging voters away from Trump and towards Harris, particularly by making voters within the examine join Trump to political violence.

This tactic is particularly prone to work with reasonable Republicans. Key races in 2022 had been decided, largely, by a small group of registered Republicans who crossed the aisle to vote towards extremists. However these similar voters had been typically completely joyful to vote for a extra regular Republican: In Georgia, for instance, incumbent Republican Gov. Brian Kemp handily defeated Democrat Stacey Abrams with help from anti-Herschel Walker Republicans.

It’s removed from apparent that these voters are steady Democrats, and Harris will want them if she desires to rebuild the coalition that powered Joe Biden to victory in 2020. As such, it will be unwise to desert essentially the most confirmed line of argument for reaching them: that democracy is at stake and due to this fact partisan grievances must take a again seat to saving the republic.

It’s maybe comprehensible why Harris and her marketing campaign seemingly selected to step away from the message, given Biden’s marketing campaign struggles even earlier than the disastrous June debate, however that skepticism can be flawed. Whereas democracy was certainly on the middle of his doomed marketing campaign, it’s laborious guilty the message for its failure. All the things we’ve seen since June means that considerations about Biden’s age made it unimaginable for any message, irrespective of how good, to interrupt via with persuadable voters.

Latest proof means that Harris can successfully make a democracy argument in a means Biden couldn’t. A just-released tutorial examine — the primary to look at the impact of democracy rhetoric on help for Harris particularly — means that she may make inroads amongst persuadable Republicans by specializing in democracy.

The examine’s authors confirmed Republican voters “democracy is on the poll”-type messaging and requested them the way it affected their political opinions. On the entire, the authors discovered, the results had been minimal and at instances even counterproductive — inflicting Republicans to dig in tougher on their partisan id. This sort of rhetoric, they conclude, “doesn’t change the attitudes that allowed Trump to problem the legitimacy of the 2020 election.”

However there was a notable exception to those dour findings: vote alternative within the 2024 election.

After a Republican viewers heard “democracy is on the poll”-type rhetoric, Harris’s help within the group greater than tripled — going from 2.3 p.c to 7.3 p.c. “The remedy induces a damaging emotional response in Republicans and successfully shames them to report help for Harris,” the examine’s authors write.

Unsurprisingly, the impact light as time handed; it’s laborious to alter somebody’s thoughts completely after a single experiment. However maybe repeating anti-authoritarian messaging via a complete marketing campaign might need extra sturdy results.

For all their considerations concerning the limits of this strategy, the authors concede that democracy messaging “has fascinating results within the short-term (a single election).”

How Harris can marketing campaign on democracy — and win

Registered Republicans aren’t the one sort of persuadable voters who matter. There’s a a lot bigger group of independents who don’t usually pay very a lot consideration to politics or particularly like both main celebration. These voters are more likely to end up in presidential elections than in midterms, and, per polling, are largely unmoved by pro-democracy rhetoric.

“Our mantra has turn out to be ‘you defend democracy by training it.’ And also you apply it by specializing in voters within the center,” says Liam Kerr, co-founder of the Democratic WelcomePAC. “A number of gettable voters — particularly at this level, after eight years — don’t imagine that stormtroopers are going to come back into the streets the day after the election.”

However no matter message you suppose this group desires — be it tough-on-the-border centrism, a liberal deal with abortion bans, or middle-class financial populism — it’s not apparent that it trades off with pro-democracy messaging.

Any good marketing campaign develops totally different messages for various teams: There’s little alternative value in focusing extra on democracy when reaching out to reasonable suburban Republicans than disaffected downscale Democrats. Even a high-profile deal with like her DNC speech can embody a fairly sized part on such a vital concern with out dragging.

Furthermore, it’s attainable to tie a number of messages collectively, particularly via overarching themes like “freedom.”

In my new e book The Reactionary Spirit, I examine how pro-democracy candidates — in the US and around the globe — beat authoritarian rivals in elections. One of the vital constant findings is that it’s attainable, and fairly efficient, to marketing campaign on democracy along side different points: displaying how anti-democratic politics impacts voters’ lives in concrete methods.

In Brazil’s 2022 presidential election, the left-wing candidate Luiz Inacio Lula da Silva linked democracy to his core message of advancing financial equality, describing it as a automobile to construct shared prosperity. In Budapest’s 2019 mayoral election, the opposition candidate Gergely Karácsony tied democracy to housing, arguing that the focus of energy within the fingers of the ruling Fidesz celebration was answerable for the federal government’s failure to handle excessive prices of dwelling.

Within the e book, I name this technique “linkage” and argue that Democrats used a model of it successfully within the midterms. In linking abortion restrictions and January 6 as shared assaults on American freedoms, they made a powerful case that Republicans had been an extremist celebration attacking cherished American values.

Harris’ “freedom” marketing campaign makes a linkage technique exceptionally simple to execute. She doesn’t want to transform her marketing campaign rhetoric or technique, however moderately spend extra effort and time explaining why the best to decide on your personal leaders is one among a number of necessary freedoms imperiled by the Trump-Vance ticket.

One other efficient avenue for making democracy really feel actual is by attacking Trump’s concepts, particularly these contained in the Mission 2025 planning doc.

Simply over every week earlier than Joe Biden dropped out, I attended a method assembly on the Heart for American Progress (CAP) — the Democratic institution’s main DC suppose tank — concerning the celebration’s Mission 2025 messaging. The information confirmed that Democrats had performed a outstanding job at each elevating consciousness of Mission 2025 and convincing People that it was a foul factor.

CAP’s researchers wished to know why, and the way Democrats may take benefit in November. So that they performed in depth interviews with voters, with a particular deal with Black and Latino voters who some polls confirmed shifting into the Trump column. As a part of this analysis, they introduced voters with a sequence of assault strains on Mission 2025 and requested them to charge which had been most and least regarding.

All 4 of the top-performing messages — those voters persistently ranked as most regarding — targeted on the menace its proposals posed to democracy. The most effective was a so-called “robust man” message, arguing that “the restructuring of presidency that Mission 2025 lays out is much like the playbook dictators and authoritarian governments have used repeatedly during the last century.”

After I spoke with the researchers once more after Biden dropped out, they argued that Harris wanted to maintain up this theme. It labored, they mentioned, as a result of it picked up on one thing necessary to many citizens — their freedom to decide on who governs them — and linked it to one thing new.

“There’s 800 pages of the identical outdated conservative nonsense in right here. However what’s so totally different is the strategy, the means to finish it — it’s the takeover. And other people freak out about that,” says Joe Radosevich, CAP’s vice chairman for campaigns and outreach.

Campaigning on democracy, in different phrases, isn’t nearly repeating the phrase “democracy” like a mantra. It’s about determining artistic methods to make the protection of democracy really feel important and related. And there are many methods for Harris to do exactly that.

finder – Easy methods to take away a file on a SMB share named “.”?

0


Overview

Incorrect specification of an argument in a commandline device (particulars beneath) has led to the creation of a file actually named “.”. I’ve already discovered that straight making an attempt to delete in Finder will set off deletion of all content material within the containing folder (thankfully this can be a non permanent folder), however nonetheless not the “.” file itself. Furthermore, no folders containing this file may be efficiently deleted, neither with Finder nor from bash/zsh shell.

How can one truly delete (or rename) such a file?


Makes an attempt to delete from zsh

For instance:

mytmp % ls -hal
whole 65
drwx------  1 consumer  employees    16K Aug 21 11:20 .
-rwx------  1 consumer  employees     0B Aug 21 09:51 .
drwx------  1 consumer  employees    16K Aug 16 16:30 ..

mytmp % ls -aB
.   .   ..

mytmp % rm -rf '.'                   
rm: "." and ".." will not be eliminated

mytmp % cd ..

scratch % rm -rf mytmp
rm: mytmp: Permission denied

scratch % sudo rm -rf mytmp
rm: mytmp: Permission denied

I’ve additionally tried variations with out -r, since actually it’s the non-directory model that’s to be deleted.

I moreover tried the suggestion of @bmike to use the inode. Whereas we are able to determine the inode, the deletion doesn’t seem to work:

scratch % ls -ila mytmp
whole 65
8056451580272514705 drwx------  1 consumer  employees  16384 Aug 21 11:20 .
8652239633868421122 -rwx------  1 consumer  employees      0 Aug 21 09:51 .
                  2 drwx------  1 consumer  employees  16384 Aug 21 11:43 ..

scratch % discover mytmp -inum 8652239633868421122 -delete

## no change
scratch % ls -ila mytmp
whole 65
8056451580272514705 drwx------  1 consumer  employees  16384 Aug 21 11:20 .
8652239633868421122 -rwx------  1 consumer  employees      0 Aug 21 09:51 .
                  2 drwx------  1 consumer  employees  16384 Aug 21 11:43 ..

Is it actually named “.”?

@nohillside proposed to delimit the listings with x and y to see if it actually is called “.”. From bash:

bash-3.2$ for i in .*; do echo x${i}y; completed
x.y
x.y
x..y

Seems so.

@fd0 instructed printing non-printing characters with cat -vet. From bash:

bash-3.2$ ls -1a | cat -vet
.$
.$
..$

Once more, appears identically named.

@nohillside Steered working a Python server from the listing and show its listing itemizing:

Directory listing from Python server is empty


Further Background

The folder is on a community Quantity, format SMB (OS X).

The device that resulted on this was haplogrep, a Java-based CLI. One can set up it by way of Conda by way of

conda set up -c conda-forge -c bioconda haplogrep

The subcommand used was haplogrep classify, which has the next choices:

mytmp % haplogrep classify

mtDNA Haplogroup Classifiction v2.4.0
https://github.com/seppinho/haplogrep-cmd
(c) Sebastian Schönherr, Hansi Weissensteiner, Lukas Forer, Dominic Pacher
[email protected]

[classify]
Lacking required choices: '--input=', '--output=', '--format='
Utilization: haplogrep classify [--chip] [--extend-report] [--rsrs]
                          [--skip-alignment-rules] [--write-fasta]
                          [--write-fasta-msa] --format=
                          [--hetLevel=] [--hits=] --in=
                          [--lineage=] [--metric=] --out=
                          [--phylotree=]
      --chip                VCF knowledge from a genotype chip
                              Default: false
      --extend-report       Add flag for a prolonged closing output
                              Default: false
      --format=     Specify enter file format: vcf, fasta or hsd
      --hetLevel= Add heteroplasmies with a stage > X from the VCF
                              file to the profile (default: 0.9)
      --hits=         Calculate finest n hits
      --in, --input=    Enter VCF, fasta or hsd file
      --lineage=   Export lineage info as dot file, n0=no
                              tree, 1=with SNPs, 2=solely construction, no SNPs
      --metric=     Specifiy different metrics (hamming or jaccard) than
                              default (kulczynski)
      --out, --output= Output file location
      --phylotree=    Specify phylotree model
      --rsrs                Use RSRS Model
                              Default: false
      --skip-alignment-rules
                            Skip mtDNA nomenclature fixes based mostly on guidelines for
                              FASTA import
                              Default: false
      --write-fasta         Write leads to fasta format
                              Default: false
      --write-fasta-msa     Write a number of sequence alignment (_MSA.fasta)
                              Default: false

I misinterpreted the “Output file location” description of the --out argument as asking for a path, main me to make use of --out . and thus leading to making a file named “.”.

Renaming

The file itself can’t be renamed in Finder or with mv, nevertheless, the containing folder may be renamed.

$75 Million Report-Breaking Ransom Paid To Cybercriminals, Say Researchers


The staggering sum of US $75 million has reportedly been paid to a ransomware gang in what’s believed to be the biggest recognized ransom cost made by a cyber assault sufferer since information started. 

Researchers at Zscaler declare in a brand new report that the record-breaking determine was paid by an undisclosed Fortune 50 firm to the Darkish Angels ransomware group. 

The reported cost virtually doubles the earlier file – $40 million paid by insurance coverage big CNA Monetary in 2021 after being locked out of its community by attackers utilizing the Phoenix Locker ransomware. 

Darkish Angels, which emerged in Might 2022, has focused a variety of industries together with healthcare, finance, authorities, and schooling. Most not too long ago it has been seen launching assaults in opposition to giant industrial, technological and telecoms corporations. 

Via its Dunghill knowledge leak web site on the darkish net, Darkish Angels claims to be “a world group of technical specialists conducting analysis within the area of knowledge safety” that’s “not excited about politics, and that’s the reason we don’t cooperate with governments and regulation enforcement companies.” 

The reality is, after all, that Darkish Angels’ manner of creating wealth is thru extortion – threatening corporations that their knowledge will probably be leaked to the world if a ransom is just not paid. 

Darkish Angels, having compromised an organization’s safety, resolve whether or not to encrypt a enterprise’s information after which, as a rule, spend days and even weeks exfiltrating huge quantities of knowledge. 

Within the instances of bigger companies which were infiltrated by the group, as much as 100 TB of knowledge could also be stolen in accordance with Zscaler’s researchers

In a high-profile incident reported by Bleeping Laptop in September 2023, Darkish Angels hit a multinational conglomerate, forcing it to close down its IT techniques, having encrypted the agency’s VMware ESXi digital machines and claimed to have stolen over 27 TB of company knowledge. 

Darkish Angels reportedly demanded a US $51 million ransom from Johnson Controls in alternate for a decryption instrument and to delete the information it had stolen. The corporate later reported in an SEC submitting that the expense of investigating and remediating the assault, in addition to losses attributable to enterprise disruption, had value it over US $27 million

Confronted with the headline of an organization paying a record-breaking US $75 million ransom cost, many companies could be questioning how they might reply when introduced with a requirement from cybercriminals. 

Admittedly, it is most likely an excellent deal simpler for a enterprise to resolve whether or not it ought to cough up tens of hundreds of thousands of {dollars} to a ransomware gang than ten thousand {dollars} – however the questions you might want to ask your self stay the identical. 

Everyone knows that the extra companies conform to pay a ransom, the extra doubtless it’s that cybercriminals will launch related assaults in opposition to others sooner or later – in addition to, maybe, your organization once more.

On the identical time, your organization could really feel it has no selection however to make the exhausting choice to pay. In any case, the choice could put the whole enterprise in danger – and put the livelihoods of your employees, companions, and maybe even shoppers in danger. 

No matter your choice, I might say that it’s important to tell regulation enforcement companies of the incident and work with them to assist them examine who is likely to be behind the assaults.

Most significantly, do not forget that paying a ransom doesn’t be certain that the safety drawback that allowed the attackers into your community within the first place now not exists. In case you don’t discover out what went incorrect – and why – and repair it, then you could possibly simply fall sufferer to additional ransomware assaults sooner or later.


Editor’s Be aware: The opinions expressed on this and different visitor writer articles are solely these of the contributor and don’t essentially mirror these of Tripwire.

Optimize your workloads with Amazon Redshift Serverless AI-driven scaling and optimization

0


The present scaling strategy of Amazon Redshift Serverless will increase your compute capability primarily based on the question queue time and scales down when the queuing reduces on the info warehouse. Nevertheless, you may must mechanically scale compute assets primarily based on elements like question complexity and information quantity to fulfill price-performance targets, regardless of question queuing. To handle this requirement, Redshift Serverless launched the synthetic intelligence (AI)-driven scaling and optimization characteristic, which scales the compute not solely primarily based on the queuing, but in addition factoring information quantity and question complexity.

On this submit, we describe how Redshift Serverless makes use of the brand new AI-driven scaling and optimization capabilities to handle widespread use circumstances. This submit additionally consists of instance SQLs, which you’ll run by yourself Redshift Serverless information warehouse to expertise the advantages of this characteristic.

Answer overview

The AI-powered scaling and optimization characteristic in Redshift Serverless gives a user-friendly visible slider to set your required steadiness between worth and efficiency. By transferring the slider, you’ll be able to select between optimized for value, balanced efficiency and value, or optimized for efficiency. Based mostly on the place you place the slider, Amazon Redshift will mechanically add or take away assets to make sure higher conduct and carry out different AI-driven optimizations like computerized materialized views and computerized desk design optimization to fulfill your chosen price-performance goal.

Price Performance Slider

The slider gives the next choices:

  • Optimized for value – Prioritizes value financial savings. Redshift makes an attempt to mechanically scale up compute capability when doing so and doesn’t incur extra expenses. And it’ll additionally try and scale down compute for decrease value, regardless of longer runtime.
  • Balanced – Affords steadiness between efficiency and value. Redshift scales for efficiency with a reasonable value improve.
  • Optimized for efficiency – Prioritizes efficiency. Redshift scales aggressively for max efficiency, probably incurring greater prices.

Within the following sections, we illustrate how the AI-driven scaling and optimization characteristic can intelligently predict your workload compute wants and scale proactively for 3 situations:

  • Use case 1 – A protracted-running advanced question. Compute scales primarily based on question complexity.
  • Use case 2 – A sudden spike in ingestion quantity (a three-fold improve, from 720 million to 2.1 billion). Compute scales primarily based on information quantity.
  • Use case 3 – An information lake question scanning giant datasets (TBs). Compute scales primarily based on the anticipated information to be scanned from the info lake. The anticipated information scan is predicted by machine studying (ML) fashions primarily based on prior historic run statistics.

Within the current auto scaling mechanism, the use circumstances don’t improve compute capability mechanically except queuing is recognized throughout the occasion.

Stipulations

To observe alongside, full the next stipulations:

  1. Create a Redshift Serverless workgroup in preview mode. For directions, see Making a preview workgroup.
  2. Whereas creating the preview group, select Efficiency and Price Controls and Value-performance goal, and regulate the slider to Optimized for efficiency. For extra data, confer with Amazon Redshift provides new AI capabilities, together with Amazon Q, to spice up effectivity and productiveness.
  3. Arrange an AWS Identification and Entry Administration (IAM) function because the default IAM function. Confer with Managing IAM roles created for a cluster utilizing the console for directions.
  4. We use TPC-DS 1TB Cloud Knowledge Warehouse Benchmark information to exhibit this characteristic. Run the SQL statements to create tables and cargo the TPC-DS 1TB information.

Use case 1: Scale compute primarily based on question complexity

The next question analyzes product gross sales throughout a number of channels akin to web sites, wholesale, and retail shops. This advanced question sometimes takes about 25 minutes to run with the default 128 RPUs. Let’s run this workload on the preview workgroup created as a part of stipulations.

When a question is run for the primary time, the AI scaling system might make a suboptimal determination relating to useful resource allocation or scaling because the system continues to be studying the question and information traits. Nevertheless, the system learns from this expertise, and when the identical question is run once more, it could possibly make a extra optimum scaling determination. Subsequently, if the question didn’t scale in the course of the first run, it’s endorsed to rerun the question. You may monitor the RPU capability used on the Redshift Serverless console or by querying the SYS_SERVERLSS_USAGE system view.

The outcomes cache is turned off within the following queries to keep away from fetching outcomes from the cache.

SET enable_result_cache_for_session TO off;
with /* TPC-DS demo question */
    ws as
    (choose d_year AS ws_sold_year, ws_item_sk,    ws_bill_customer_sk
     ws_customer_sk,    sum(ws_quantity) ws_qty,    sum(ws_wholesale_cost) ws_wc,
        sum(ws_sales_price) ws_sp   from web_sales   left be part of web_returns on
     wr_order_number=ws_order_number and ws_item_sk=wr_item_sk   be part of date_dim
     on ws_sold_date_sk = d_date_sk   the place wr_order_number is null   group by
     d_year, ws_item_sk, ws_bill_customer_sk   ),
    cs as  
    (choose d_year AS cs_sold_year,
     cs_item_sk,    cs_bill_customer_sk cs_customer_sk,    sum(cs_quantity) cs_qty,
        sum(cs_wholesale_cost) cs_wc,    sum(cs_sales_price) cs_sp   from catalog_sales
       left be part of catalog_returns on cr_order_number=cs_order_number and cs_item_sk=cr_item_sk
       be part of date_dim on cs_sold_date_sk = d_date_sk   the place cr_order_number is
     null   group by d_year, cs_item_sk, cs_bill_customer_sk   ),
    ss as  
    (choose
     d_year AS ss_sold_year, ss_item_sk,    ss_customer_sk,    sum(ss_quantity)
     ss_qty,    sum(ss_wholesale_cost) ss_wc,    sum(ss_sales_price) ss_sp
       from store_sales left be part of store_returns on sr_ticket_number=ss_ticket_number
     and ss_item_sk=sr_item_sk   be part of date_dim on ss_sold_date_sk = d_date_sk
       the place sr_ticket_number is null   group by d_year, ss_item_sk, ss_customer_sk
       ) 
       
       choose 
       ss_customer_sk,spherical(ss_qty/(coalesce(ws_qty+cs_qty,1)),2)
     ratio,ss_qty store_qty, ss_wc store_wholesale_cost, ss_sp store_sales_price,
    coalesce(ws_qty,0)+coalesce(cs_qty,0) other_chan_qty,coalesce(ws_wc,0)+coalesce(cs_wc,0)
     other_chan_wholesale_cost,coalesce(ws_sp,0)+coalesce(cs_sp,0) other_chan_sales_price
    from ss left be part of ws on (ws_sold_year=ss_sold_year and ws_item_sk=ss_item_sk
     and ws_customer_sk=ss_customer_sk)left be part of cs on (cs_sold_year=ss_sold_year
     and cs_item_sk=cs_item_sk and cs_customer_sk=ss_customer_sk)the place coalesce(ws_qty,0)>0
     and coalesce(cs_qty, 0)>0 order by   ss_customer_sk,  ss_qty desc, ss_wc
     desc, ss_sp desc,  other_chan_qty,  other_chan_wholesale_cost,  other_chan_sales_price,
      spherical(ss_qty/(coalesce(ws_qty+cs_qty,1)),2);

When the question is full, run the next SQL to seize the beginning and finish occasions of the question, which can be used within the subsequent question:

choose query_id,query_text,start_time,end_time, elapsed_time/1000000.0 duration_in_seconds
from sys_query_history
the place query_text like '%TPC-DS demo question%'
and query_text not like '%sys_query_history%'
order by start_time desc

Let’s assess the compute scaled in the course of the previous start_time and end_time interval. Substitute start_time and end_time within the following question with the output of the previous question:

choose * from sys_serverless_usage
the place end_time >= 'start_time'
and end_time <= DATEADD(minute,1,'end_time')
order by end_time asc

-- Instance
--select * from sys_serverless_usage
--where end_time >= '2024-06-03 00:17:12.322353'
--and end_time <= DATEADD(minute,1,'2024-06-03 00:19:11.553218')
--order by end_time asc

The next screenshot exhibits an instance output.

Use Case 1 output

You may discover the rise in compute over the length of this question. This demonstrates how Redshift Serverless scales primarily based on question complexity.

Use case 2: Scale compute primarily based on information quantity

Let’s contemplate the web_sales ingestion job. For this instance, your day by day ingestion job processes 720 million data and completes in a mean of two minutes. That is what you ingested within the prerequisite steps.

As a consequence of some occasion (akin to month finish processing), your volumes elevated by thrice and now your ingestion job must course of 2.1 billion data. In an current scaling strategy, this may improve your ingestion job runtime except the queue time is sufficient to invoke extra compute assets. However with AI-driven scaling, in efficiency optimized mode, Amazon Redshift mechanically scales compute to finish your ingestion job inside traditional runtimes. This helps defend your ingestion SLAs.

Run the next job to ingest 2.1 billion data into the web_sales desk:

copy web_sales from 's3://redshift-downloads/TPC-DS/2.13/3TB/web_sales/' iam_role default gzip delimiter '|' EMPTYASNULL area 'us-east-1';

Run the next question to match the length of ingesting 2.1 billion data and 720 million data. Each ingestion jobs accomplished in roughly the same time, regardless of the three-fold improve in quantity.

choose query_id,table_name,data_source,loaded_rows,length/1000000.0 duration_in_seconds , start_time,end_time
from sys_load_history
the place
table_name="web_sales"
order by start_time desc

Run the next question with the beginning occasions and finish occasions from the earlier output:

choose * from sys_serverless_usage
the place end_time >= 'start_time'
and end_time <= DATEADD(minute,1,'end_time')
order by end_time asc

The next is an instance output. You may discover the rise in compute capability for the ingestion job that processes 2.1 billion data. This illustrates how Redshift Serverless scaled primarily based on information quantity.

Use Case 2 Output

Use case 3: Scale information lake queries

On this use case, you create exterior tables pointing to TPC-DS 3TB information in an Amazon Easy Storage Service (Amazon S3) location. Then you definately run a question that scans a big quantity of information to exhibit how Redshift Serverless can mechanically scale compute capability as wanted.

Within the following SQL, present the ARN of the default IAM function you connected within the stipulations:

-- Create exterior schema
create exterior schema ext_tpcds_3t
from information catalog
database ext_tpcds_db
iam_role ''
create exterior database if not exists;

Create exterior tables by working DDL statements within the following SQL file. You must see seven exterior tables within the question editor below the ext_tpcds_3t schema, as proven within the following screenshot.

External Tables

Run the next question utilizing exterior tables. As talked about within the first use case, if the question didn’t scale in the course of the first run, it’s endorsed to rerun the question, as a result of the system could have discovered from the earlier expertise and might probably present higher scaling and efficiency for the next run.

The outcomes cache is turned off within the following queries to keep away from fetching outcomes from the cache.

SET enable_result_cache_for_session TO off;

with /* TPC-DS demo information lake question */

ws as
(choose d_year AS ws_sold_year, ws_item_sk, ws_bill_customer_sk
ws_customer_sk,    sum(ws_quantity) ws_qty,    sum(ws_wholesale_cost) ws_wc,
sum(ws_sales_price) ws_sp   from ext_tpcds_3t.web_sales   left be part of ext_tpcds_3t.web_returns on
wr_order_number=ws_order_number and ws_item_sk=wr_item_sk   be part of ext_tpcds_3t.date_dim
on ws_sold_date_sk = d_date_sk   the place wr_order_number is null   group by
d_year, ws_item_sk, ws_bill_customer_sk   ),

cs as
(choose d_year AS cs_sold_year,
cs_item_sk,    cs_bill_customer_sk cs_customer_sk,    sum(cs_quantity) cs_qty,
sum(cs_wholesale_cost) cs_wc,    sum(cs_sales_price) cs_sp   from ext_tpcds_3t.catalog_sales
left be part of ext_tpcds_3t.catalog_returns on cr_order_number=cs_order_number and cs_item_sk=cr_item_sk
be part of ext_tpcds_3t.date_dim on cs_sold_date_sk = d_date_sk   the place cr_order_number is
null   group by d_year, cs_item_sk, cs_bill_customer_sk   ),

ss as
(choose
d_year AS ss_sold_year, ss_item_sk,    ss_customer_sk,    sum(ss_quantity)
ss_qty,    sum(ss_wholesale_cost) ss_wc,    sum(ss_sales_price) ss_sp
from ext_tpcds_3t.store_sales left be part of ext_tpcds_3t.store_returns on sr_ticket_number=ss_ticket_number
and ss_item_sk=sr_item_sk   be part of ext_tpcds_3t.date_dim on ss_sold_date_sk = d_date_sk
the place sr_ticket_number is null   group by d_year, ss_item_sk, ss_customer_sk)

SELECT           ss_customer_sk,spherical(ss_qty/(coalesce(ws_qty+cs_qty,1)),2)
ratio,ss_qty store_qty, ss_wc store_wholesale_cost, ss_sp store_sales_price,
coalesce(ws_qty,0)+coalesce(cs_qty,0) other_chan_qty,coalesce(ws_wc,0)+coalesce(cs_wc,0)    other_chan_wholesale_cost,coalesce(ws_sp,0)+coalesce(cs_sp,0) other_chan_sales_price
FROM ss left be part of ws on (ws_sold_year=ss_sold_year and ws_item_sk=ss_item_sk and ws_customer_sk=ss_customer_sk)left be part of cs on (cs_sold_year=ss_sold_year and cs_item_sk=cs_item_sk and cs_customer_sk=ss_customer_sk)
the place coalesce(ws_qty,0)>0
and coalesce(cs_qty, 0)>0
order by   ss_customer_sk,  ss_qty desc, ss_wc desc, ss_sp desc,  other_chan_qty,  other_chan_wholesale_cost,  other_chan_sales_price,     spherical(ss_qty/(coalesce(ws_qty+cs_qty,1)),2);

Evaluate the full elapsed time of the question. You want the start_time and end_time from the outcomes to feed into the subsequent question.

choose query_id,query_text,start_time,end_time, elapsed_time/1000000.0 duration_in_seconds
from sys_query_history
the place query_text like '%TPC-DS demo information lake question%'
and query_text not like '%sys_query_history%'
order by start_time desc

Run the next question to see how compute scaled in the course of the previous start_time and end_time interval. Substitute start_time and end_time within the following question from the output of the previous question:

choose * from sys_serverless_usage
the place end_time >= 'start_time'
and end_time <= DATEADD(minute,1,'end_time')
order by end_time asc

The next screenshot exhibits an instance output.

Use Case 3 Output

The elevated compute capability for this information lake question exhibits that Redshift Serverless can scale to match the info being scanned. This demonstrates how Redshift Serverless can dynamically allocate assets primarily based on question wants.

Concerns when selecting your price-performance goal

You should use the price-performance slider to decide on your required price-performance goal in your workload. The AI-driven scaling and optimizations present holistic optimizations utilizing the next fashions:

  • Question prediction fashions – These decide the precise useful resource wants (reminiscence, CPU consumption, and so forth) for every particular person question
  • Scaling prediction fashions – These predict how the question would behave on totally different capability sizes

Let’s contemplate a question that takes 7 minutes and prices $7. The next determine exhibits the question runtimes and value with no scaling.

Scaling Type Example

A given question may scale in a couple of other ways, as proven under. Based mostly on the price-performance goal you selected on the slider, AI-driven scaling predicts how the question trades off efficiency and value, and scales it accordingly.

Scaling Types

The slider choices yield the next outcomes:

  • Optimized for value – While you select Optimized for value, the warehouse scales up if there is no such thing as a extra value or lesser prices to the consumer. Within the previous instance, the superlinear scaling strategy demonstrates this conduct. Scaling will solely happen if it may be performed in a cheap method in accordance with the scaling mannequin predictions. If the scaling fashions predict that cost-optimized scaling isn’t potential for the given workload, then the warehouse gained’t scale.
  • Balanced – With the Balanced choice, the system will scale in favor of efficiency and there can be a price improve, however it is going to be a restricted improve in value. Within the previous instance, the linear scaling strategy demonstrates this conduct.
  • Optimized for efficiency – With the Optimized for efficiency choice, the system will scale in favor of efficiency although the prices are greater and non-linear. Within the previous instance, the sublinear scaling strategy demonstrates this conduct. The nearer the slider place is to the Optimized for efficiency place, the extra sublinear scaling is permitted.

The next are extra factors to notice:

  • The worth-performance slider choices are dynamic and they are often modified anytime. Nevertheless, the influence of those adjustments won’t be realized instantly. The influence of that is efficient because the system learns easy methods to scale the present workload and any extra workloads higher.
  • The worth-performance slider choices, Max capability and Max RPU-hours are designed to work collectively. Max capability and Max RPU-hours are the controls to restrict most RPUs the info warehouse allowed to scale and most RPU hours allowed to eat respectively. These controls are at all times honored and enforced whatever the settings on the price-performance goal slider.
  • The AI-driven scaling and optimization characteristic dynamically adjusts compute assets to optimize question runtime velocity whereas adhering to your price-performance necessities. It considers elements akin to question queueing, concurrency, quantity, and complexity. The system can both run queries on a compute useful resource with decrease concurrent queries or spin up extra compute assets to keep away from queueing. The aim is to offer the most effective price-performance steadiness primarily based in your decisions.

Monitoring

You may monitor the RPU scaling within the following methods:

  • Evaluate the RPU capability used graph on the Amazon Redshift console.
  • Monitor the ComputeCapacity metric below AWS/Redshift-Serverless and Workgroup in Amazon CloudWatch.
  • Question the SYS_QUERY_HISTORY view, offering the precise question ID or question textual content to establish the time interval. Use this time interval to question the SYS_SERVERLSS_USAGE system view to seek out the compute_capacity The compute_capacity area will present the RPUs scaled in the course of the question runtime.

Confer with Configure monitoring, limits, and alarms in Amazon Redshift Serverless to maintain prices predictable for the step-by-step directions on utilizing these approaches.

Clear up

Full the next steps to delete the assets you created to keep away from surprising prices:

  1. Delete the Redshift Serverless workgroup.
  2. Delete the Redshift Serverless related namespace.

Conclusion

On this submit, we mentioned easy methods to optimize your workloads to scale primarily based on the adjustments in information quantity and question complexity. We demonstrated an strategy to implement extra responsive, proactive scaling with the AI-driven scaling characteristic in Redshift Serverless. Do that characteristic in your atmosphere, conduct a proof of idea in your particular workloads, and share your suggestions with us.


In regards to the Authors

Satesh Sonti is a Sr. Analytics Specialist Options Architect primarily based out of Atlanta, specialised in constructing enterprise information platforms, information warehousing, and analytics options. He has over 19 years of expertise in constructing information property and main advanced information platform packages for banking and insurance coverage shoppers throughout the globe.

Ashish Agrawal is a Principal Product Supervisor with Amazon Redshift, constructing cloud-based information warehouses and analytics cloud companies. Ashish has over 25 years of expertise in IT. Ashish has experience in information warehouses, information lakes, and platform as a service. Ashish has been a speaker at worldwide technical conferences.

Davide Pagano is a Software program Growth Supervisor with Amazon Redshift primarily based out of Palo Alto, specialised in constructing cloud-based information warehouses and analytics cloud companies options. He has over 10 years of expertise with databases, out of which 6 years of expertise tailor-made to Amazon Redshift.

Containers on the Edge with David Aronchick


Massive datasets require massive computational sources to course of that knowledge. Extra regularly, the place you course of that knowledge geographically will be simply as vital as the way you course of it.

Expanso supplies job execution infrastructure that runs jobs the place knowledge resides, to assist cut back latency and enhance safety and knowledge governance.

David Aronchick is the CEO of Expanso. He beforehand labored at Google on the Kubernetes workforce, which influenced his determination to begin Expanso. David joins the present to speak about his firm.

This episode is hosted by Lee Atchison. Lee Atchison is a software program architect, creator, and thought chief on cloud computing and utility modernization. His best-selling ebook, Architecting for Scale (O’Reilly Media), is a vital useful resource for technical groups trying to preserve excessive availability and handle danger of their cloud environments.

Lee is the host of his podcast, Fashionable Digital Enterprise, a fascinating and informative podcast produced for folks trying to construct and develop their digital enterprise with the assistance of contemporary functions and processes developed for as we speak’s fast-moving enterprise setting. Pay attention at mdb.fm. Observe Lee at softwarearchitectureinsights.com, and see all his content material at leeatchison.com.

Sponsors

This episode of Software program Engineering Day by day is delivered to you by SquiggleConf: a convention for glorious internet dev tooling.

SquiggleConf is bringing collectively the admirers, contributors, and maintainers of the most recent and best in developer instruments to the Boston New England Aquarium on October third and 4th. That’s proper, an internet dev tooling convention at an aquarium.

SquiggleConf will present you the right way to supercharge your internet builders and groups. It’ll function talks and workshops from the builders across the TypeScript, Rust, and Zig instruments that make the squiggles you rely on day by day. You’ll emerge with the most effective strategies to supercharge your initiatives — whether or not you’re already a tooling wizard or attempting to grow to be one.

See squiggleconf.com for more information. Enter promo code SED at checkout for 10% off.

Digital entrepreneurs, this one’s for you. Introducing, Wix Studio, the net platform for businesses and enterprises. So, right here are some things you are able to do in thirty seconds or much less while you handle initiatives on Wix Studio:

Work in sync along with your workforce on one canvas

Reuse templates, widgets and sections throughout websites

Create a shopper package for seamless handovers

And leverage best-in-class search engine marketing defaults throughout all of your Wix websites

The listing retains going! Step into Wix Studio to see extra.